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ABSTRACT 


This  thesis  defines  a  campaign  of  experimentation  to  guide  UTACC  development 
from  concept  to  reality.  It  also  applies  design  methodologies  to  reduce  costs  and  increase 
the  quality,  effectiveness,  and  speed  of  UTACC ’s  development.  UTACC  is  a  system  of 
systems  that  teams  Marines  with  unmanned  robotic  systems  to  reduce  the  Marine’s 
cognitive  load  and  enhance  mission  accomplishment.  Bringing  UTACC  from  concept  to 
reality  requires  extensive  experimentation,  but  prior  to  this  thesis  no  experimentation  plan 
has  existed. 

A  series  of  UTACC  theses  have  been  written  starting  with  a  CONOPS.  Then 
theses  red-celled  the  CONOPS,  explored  Coactive  Design  methodology,  analyzed  UAV 
alternatives,  and  generated  measures  of  effectiveness  and  performance  for  the  system. 
Using  information  learned  from  the  previous  theses,  the  campaign  of  experimentation 
described  in  this  thesis  identifies  key  developmental  relationships,  associates  measures 
with  them,  and  organizes  them  in  an  incremental  order.  This  thesis  also  emphasizes 
Coactive  Design  and  Model  Driven  Software  Development  to  reduce  cost  and  improve 
the  quality  and  flexibility  of  the  system.  The  goal  of  the  campaign  is  to  provide  a  plan  to 
develop  a  robust,  cost-efficient  system  that  Marines  can  use  as  a  part  of  their  team  to 
increase  victory  on  the  battlefield. 
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EXECUTIVE  SUMMARY 


Rapidly  evolving  technology  has  created  information  overload  for  decision 
makers.  They  are  expected  to  pull  specific  and  relevant  information  from  a  vast  sea  of 
data  and  then  make  decisions  that  impact  Marines  on  the  battlefield.  The  abundance  of 
information  can  overwhelm  warfighters  and  lead  to  degraded  mission  performance  (Rice, 
Keim,  &  Chhabra,  2015,  p.  3).  The  Unmanned  Tactical  Autonomous  Control  and 
Collaboration  (UTACC)  system  is  designed  to  enhance  mission  accomplishment  while 
reducing  the  information  load  on  the  warfighter.  UTACC  consists  of  Marines,  an 
unmanned  ground  component,  and  an  unmanned  aerial  component  acting  as  a  team  to 
accomplish  future  operations. 

This  thesis  is  the  sixth  in  a  series  of  theses  that  support  the  Marine  Corps 
Warfighting  Laboratory’s  (MCWL)  development  of  the  UTACC  system.  It  describes 
how  a  campaign  of  experimentation  will  help  system  developers  advance  UTACC  from 
concept  to  functional  system.  This  thesis  also  uses  design  methodologies  to  facilitate 
efficient  and  effective  system  development.  The  campaign  of  experimentation  satisfies 
the  system  engineering  criteria  of  detail  design  and  development. 

The  campaign  of  experimentation  is  based  on  the  concept  of  operations 
(CONOPS)  that  focuses  system  development.  The  campaign  follows  the  Command  and 
Control  Research  Program’s  guiding  principles  of  variety  and  replication.  Their  use 
ensures  that  a  successful  system  is  developed  in  a  comprehensive  and  incremental  way. 
Seven  Critical  Operational  Issues  (COI)  provided  in  the  planning  worksheet  attached  in 
the  appendixes  are  the  foundation  for  the  hypotheses  driving  experimentation. 

The  campaign  of  experimentation  uses  Limited  Technical  Assessments  (LTA)  to 
organize  experimentation  into  stages.  Two  LTAs  are  scheduled  to  take  place  each  year. 
This  allows  a  six  month  period  for  developers  to  conduct  their  own  experimentation  and 
address  any  issues  that  arise.  MOEs  and  MOPs  provide  quantifiable  standards  to  evaluate 
newly  developed  system  capabilities.  Using  them  as  entrance  and  exit  criteria  ensures 


that  replication  occurs  throughout  the  experimentation  process  and  that  the  needs  of  the 
Marine  Corps  are  met. 

Coactive  design  is  one  of  the  proposed  design  methodologies  for  UTACC 
developers.  It  focuses  on  human-machine  interaction.  The  Marine  Corps  planning  process 
is  a  framework  for  the  UTACC  coactive  design  process.  Coactive  design  is  used  to 
identify  the  different  variables  associated  with  interdependence,  tasks  to  be  completed, 
and  the  relationship  between  the  two.  It  is  a  unique  combination  of  waterfall  and  spiral 
development  models.  The  waterfall  attributes  of  the  process  make  it  easier  to  follow  and 
execute  while  the  spiral  model  attributes  facilitate  adaptation  throughout  the  design 
process  (Satzinger,  Jackson,  &  Burd,  2012,  pp.  228,  230).  The  simplicity  and  flexibility 
of  the  coactive  design  method  is  a  tremendous  advantage  for  the  UTACC  development 
team.  They  can  quickly  identify  interdependence  variables  and  either  add,  subtract,  or 
modify  variables  throughout  the  process. 

Model  driven  software  development  (MDSD)  is  the  second  design  methodology 
recommended  in  this  thesis.  MDSD  fits  the  DOD  standard  for  software  development 
outlined  in  DOD  publication  5000.02  Operation  of  the  Defense  Acquisition  System.  The 
goals  of  MDSD  are  to  increase  development  speed,  improve  the  quality  of  the  software 
created,  improve  software  maintenance,  increase  reusability,  manage  the  complexity  of 
system,  and  increase  interoperability  (Stahl  et  al.,  2006,  p.  13-14).  To  accomplish  these 
goals,  software  developers  utilize  models.  The  collection  of  models  makes  up  an 
architecture.  The  architecture  defines  the  system  or  system  of  systems  and  serves  as  a 
blueprint  that  software  development  teams  can  use  as  a  foundation  to  create  an 
application  (Stahl  et  al.,  2006,  p.  22).  The  more  detailed  the  architecture,  the  more 
thorough  a  blueprint  is  created,  and  the  more  efficient  the  software  developers  are 
because  they  can  copy  source  coding  rather  than  create  new  coding  for  a  function. 

UTACC  is  a  unique,  innovative  system  that  teams  a  Marine  with  a  machine  to 
reduce  the  Marines’  cognitive  load  and  enhance  mission  success.  Going  from  concept  to 
reality  requires  robust,  incremental  experimentation.  This  thesis  proposes  a  campaign  of 
experimentation  to  accomplish  exactly  that.  It  focuses  resources  and  the  efforts  of 
developers  to  create  a  system  that  serves  as  teammate  on  the  battlefield  instead  of  a  tool. 
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I. 


INTRODUCTION 


A.  UTACC  VISION 

There  are  a  large  number  of  sensors  and  technologies  used  for  war  that  are 
designed  to  increase  mission  accomplishment  for  the  warfighter.  These  technologies  have 
created  information  overload  for  the  decision  maker.  Decision  makers  must  pull  specific 
information  from  a  vast  pool  before  making  decisions  that  impact  lives  on  the  battlefield. 
This  abundance  of  information  can  easily  overwhelm  the  warfighter  and  lead  to 
unintentional  degraded  mission  performance  (Rice,  Keim,  &  Chhabra,  2015,  p.  3). 

As  stated  by  Rice  et  al.  (2015),  the  Unmanned  Tactical  Autonomous  Control  and 
Collaboration  (UTACC)  system’s  purpose  is  to  enhance  mission  accomplishment  while 
reducing  the  warfighter’s  information  overload.  Collaborative  autonomy  accomplishes 
that  goal.  The  system  will  consist  of  a  team  member,  an  unmanned  ground  component, 
and  an  unmanned  aerial  component  acting  as  a  team  to  accomplish  future  operations. 
System  developers  analyze  operational  context,  possible  missions  with  associated  tasks, 
human  system  integration  (HSI)  factors,  and  data  exchange  requirements  to  understand 
the  inherent  complexities  of  the  system  and  plan  a  way  forward.  The  Marine  Corps  will 
need  to  innovate  new  technology  to  create  a  UTACC  system  that  can  function  as  an 
integral  part  of  the  team. 

This  thesis  is  the  sixth  in  a  series  of  theses  that  discuss  the  Marine  Corps 
Warfighting  Laboratory’s  (MCWL)  development  of  the  UTACC  system.  This  thesis  is 
similar  to  the  previous  five  theses  because  UTACC  development  utilizes  the  incremental 
design  process.  MCWL’s  mission  is  to  rigorously  explore  and  assess  Marine  Corps 
service  concepts  using  war  gaming,  concept-based  experimentation,  technology 
assessments,  and  analysis.  In  addition  to  assessing  service  concepts,  they  aim  to  inform 
force  development  by  validating,  modifying,  or  rejecting  concepts’  viability  while 
identifying  capability  gaps  and  opportunities  (“MCWL,”  n.d.,  mission).  The  first  thesis  in 
this  series  developed  a  concept  of  operations  for  UTACC.  The  second  thesis  conducted  a 
“Red  Team”  critique  of  the  concept  of  operations  (CONOPS).  The  third  thesis  explored 
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Marine  and  machine  interdependence.  The  fourth  thesis  used  CONOPS  to  develop 
specific  measures  of  effectiveness  (MOE)  and  measures  of  performance  (MOP)  for 
UTACC.  The  fifth  thesis  analyzes  MCWL  alternatives  for  an  appropriate  UTACC 
unmanned  air  vehicle.  This  thesis  adapts  the  fourth  thesis’s  development  of  UTACC 
MOEs  and  MOPs  to  propose  a  coherent  campaign  of  experimentation  to  ensure  that 
UTACC  focuses  on  the  most  important  operational  and  technical  concepts. 

B.  RESEARCH  QUESTION 

This  thesis  proposes  a  campaign  of  experimentation  that  system  developers  can 
use  to  advance  UTACC  from  concept  to  functional  system.  This  thesis  will  answer 
the  following  questions:  1)  What  is  a  campaign  of  experimentation  and  how  should 
it  apply  to  UTACC?  2)  What  are  the  key  operational  and  technical  elements  of  UTACC? 

3)  How  can  one  use  MOEs  and  MOPs  to  determine  if  those  elements  have  been  met? 

4)  How  should  the  campaign  of  experimentation  arrange  the  order  of  these  elements? 

5)  What  are  the  associated  entrance  and  exit  criteria  required  to  move  from  one  element 
focus  area  to  the  next?  6)  Can  some  elements  be  worked  in  parallel?  7)  If  so,  which  ones? 
These  questions  form  the  bedrock  for  understanding  what  a  campaign  of  experimentation 
is,  how  UTACC  developers  can  utilize  it,  and  how  the  process  of  system  development 
can  occur.  Successful  UTACC  system  development  requires  the  use  of  funds,  new 
technologies,  and  a  variety  of  novel  innovations  from  diverse  organizations.  An 
incremental  guide  for  experimentation  will  allow  the  Marine  Corps  to  effectively  focus 
the  attention,  skills,  and  resources  needed  to  create  UTACC  (Alberts  &  Hayes,  2005, 
p.  63).  The  campaign  of  experimentation  proposed  in  this  thesis  will  attempt  to 
accomplish  just  that. 

C.  THESIS  ORGANIZATION 

The  thesis  is  organized  into  four  chapters.  The  introduction  explains  how  this 
thesis  falls  within  the  UTACC  thesis  series  and  uses  content  from  the  previous  theses. 
The  introduction  explains  the  UTACC  vision  of  creating  a  system  that  teams  unmanned 
ground  and  aerial  vehicles  with  Marines  to  reduce  their  cognitive  load  in  combat.  The 
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introduction  introduces  the  research  questions  that  guided  thesis  research  and  describes 
the  intended  impact  of  the  thesis  on  UTACC  system  development. 

Chapter  II,  the  literature  review,  summarizes  background  information  regarding 
UTACC  concepts  and  explains  the  intended  impact  of  the  system  on  the  battlefield.  The 
literature  review  explores  autonomy  in  depth,  a  critical  aspect  of  UTACC.  It  defines 
autonomy,  the  levels  of  autonomy,  and  the  Department  of  Defense’s  stance  regarding 
autonomy  and  the  potential  benefits  autonomy  will  offer  in  future  conflicts.  The  chapter 
then  explains  measures  of  effectiveness  and  performance.  Measures  of  effectiveness  and 
performance  are  standards  for  the  operation  of  a  system.  They  focus  experimentation 
efforts  by  functioning  as  entrance  and  exit  criteria  for  those  experiments.  The  chapter 
then  describes  of  a  campaign  of  experimentation  and  its  various  parts  in  detail.  The 
campaign  of  experimentation  is  a  system  development  roadmap  designed  to  be  a 
proactive,  incremental  guide  to  focus  attention  and  resources  (Alberts  &  Hayes,  2005, 
p.  63).  The  last  two  sub-sections  of  the  literature  review  introduce  different  design 
methodologies  developers  can  use  to  increase  the  speed,  quality,  and  flexibility  of  the 
system  throughout  the  development  process. 

Chapter  III  explains  key  elements  and  design  methodologies  of  the  UTACC 
campaign  of  experimentation.  The  chapter  explains  and  expands  upon  concepts 
introduced  during  the  latter  half  of  the  literature  review.  The  third  chapter  pulls  heavily 
from  the  fourth  and  fifth  theses  in  the  series.  The  chapter  highlights  the  importance  of 
measures  of  effectiveness  and  performance,  interdependence,  and  the  coactive  design 
process  because  they  ensure  a  quality  product  is  delivered  to  the  warfighter. 

The  fourth  chapter  of  the  current  research  moves  away  from  the  conceptual 

aspects  of  a  campaign  of  experimentation  and  focuses  on  the  practical  application  of  the 

campaign.  The  chapter  explains  the  logistical  components  of  a  recommended  campaign 

of  experimentation.  The  thesis  includes  a  planning  template  for  limited  technical 

assessments  and  recommends  a  plan  for  future  assessments.  The  chapter  also  lists 

entrance  and  exit  criteria  for  each  stage  of  the  campaign.  The  chapter  assesses  which 

development  efforts  can  be  run  in  parallel  to  save  system  developers  time.  Finally,  the 

chapter  summarizes  the  thesis  and  recommends  future  areas  of  research  for  UTACC.  The 
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fourth  chapter  will  briefly  describe  potential  research  areas  to  orient  developers  toward 
future  development.  An  appendix  containing  supporting  documentation  is  also  included. 


D.  SECTION  SUMMARY 

UTACC  is  a  complex  array  of  mature  and  developing  technologies  that  will  need 
to  work  together  and  in  tandem  with  humans  to  accomplish  a  mission.  Significant 
advancement  in  autonomy  and  additional  technology  is  required  before  unmanned 
systems  are  capable  of  functioning  as  Marine  teammates.  This  thesis  explains  how 
various  concepts  and  methodologies  can  enhance  the  development  of  UTACC.  The  thesis 
is  a  launching  point  for  development  teams  and  is  intended  to  generate  the  conversation 
and  debate  needed  to  move  UTACC  from  a  concept  to  a  team  member. 
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II.  LITERATURE  REVIEW 


A.  BACKGROUND 

Unmanned  Tactical  Autonomous  Collaboration  and  Control  (UTACC)  was 
developed  to  reduce  Marines’  cognitive  load.  UTACC  developers  will  increase  the 
autonomy  of  unmanned  ground  and  aerial  systems  to  change  the  relationship  between 
human  and  machine.  Currently,  unmanned  system  operators  use  a  remote  control  to 
provide  direct  input  to  the  system,  called  “human  in  the  loop”  operation  (Rice,  Keim,  & 
Chhabra,  2015,  p.  12).  UTACC’s  increased  autonomy  places  the  operator  in  a 
supervisory  role  over  the  system,  a  status  called  “human  on  the  loop”  (Rice  et  al.,  2015, 
p.  12).  Marines  give  the  machine  mission  parameters,  intent,  and  tasks  and  afterwards  the 
machine  executes  those  tasks  autonomously.  When  a  task  is  complete  or  a  critical 
decision  point  is  reached,  the  system  notifies  the  human  operator  to  receive 
acknowledgement  or  input  before  continuing  with  its  mission.  Putting  the  human  “on  the 
loop”  allows  the  warfighter  to  focus  on  warfighting  tasks.  The  push  and  pull  of 
information  between  human  and  machine  facilitates  a  teamwork  relationship  between 
man  and  machine.  The  UTACC  program  is  developing  software  for  unmanned 
systems  that  develops  collaboration  between  systems  and  Marines  (Gelhaus,  2015,  p.  7). 
To  do  this,  system  developers  will  utilize  “agent-based  reasoning  and  semantic 
technologies  to  plan  an  optimized  method  to  complete  a  task(s),  goal(s),  or  performance 
measure(s);  then  set  off  to  accomplish  those  with  (or  without)  human  partners  (vice 
operators)”  (Gelhaus,  2015,  p.  7). 

To  date,  there  have  been  two  limited  technical  assessments  to  demonstrate  and 
test  the  capabilities  of  the  UTACC  system.  The  first  limited  technical  assessment  (LTA) 
occurred  on  26  February  2015  and  the  second  on  18-22  April  2016;  representatives  from 
the  Center  for  Naval  Analysis  (CNA)  observed  both  and  wrote  reports  (Gelhaus,  2015, 
p.  4).  The  first  LTA  was  a  proof  of  concept  (POC)  and  occurred  at  Carnegie  Mellon 
University’s  (CMU)  campus  in  Pittsburgh,  Pennsylvania.  The  POC  occurred  in  four 
stages.  The  first  demonstrated  current  Marine  cognitive  load  and  how  much  time  is 
required  to  accomplish  a  mission  while  operating  a  legacy  Dragon  Runner  unmanned 
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ground  vehicle  (Gelhaus,  2015,  p.  14).  The  second  stage  of  the  POC  tested  an  unmanned 
ground  vehicle  developed  by  CMU.  The  focus  of  this  part  of  the  POC  was  mobility,  3D 
mapping,  resource  utilization,  sensor  capabilities,  system  prompts,  and  machine 
diagnostics  (Gelhaus,  2015,  p.  15).  The  third  stage  tested  autonomous  collaboration 
between  CMU’s  unmanned  ground  and  aerial  vehicles.  The  vehicles  worked  together 
using  a  shared  3D  map  to  find  a  green  pad  on  a  desk  oriented  so  the  ground  vehicle  could 
not  “see”  it  with  its  onboard  sensors  (Gelhaus,  2015,  p.  17).  The  final  stage  of  the  POC 
tested  the  autonomous  control  of  the  CMU  system  and  its  ability  to  adjust  to  changes 
in  its  environment.  It  demonstrated  how  much  time  an  operator  spends  accomplishing 
tasks  without  the  system  as  compared  to  how  much  time  is  spent  when  an  operator 
uses  the  system.  Additionally,  the  final  stage  tested  how  the  system  reacted  to 
environmental  changes  (Gelhaus,  2015,  p.  18).  CNA  concluded  that  the  autonomous 
systems  were  faster  at  accomplishing  their  tasks  than  their  human-operated  counterpart. 
Although  CNA  recommended  further  experimentation,  it  ultimately  concluded  that 
unmanned  autonomous  systems  reduced  Marines’  cognitive  load  (Gelhaus,  2015,  p.  25). 

LTA  2  took  place  from  18-22  April  2016  in  Ellis  Hall  aboard  Marine  Corps  Base 
Quantico.  It  tested  the  UTACC  hardware  and  its  collaborative  capabilities  in  a  controlled 
environment.  The  experiment  took  place  in  a  mock  village  with  adjacent  forest,  river,  and 
river  crossing  point  inside  the  auditorium  at  Ellis  Hall.  LTA  planners  prepared  eight 
scenarios  to  test  UTACC,  but  only  two  were  executed  because  software  and  hardware 
issues  left  little  time  for  the  rest.  The  first  scenario  tested  the  system’s  ability  to 
autonomously  build  a  3D  model  of  the  village  and  its  surrounding  area  using  data  from 
the  UGV  and  UAV.  During  the  second  scenario,  the  system  searched  for  a  high-value 
target  while  using  and  improving  the  map  developed  in  the  first  scenario.  The  system 
used  facial  recognition  software  to  identify  the  target.  When  either  the  UGV  or  the  UAV 
found  the  target,  the  system  requested  confirmation  from  the  Marine  on  the  loop.  When 
Marines  gave  confirmation  and  authorized  target  engagement,  the  system  requested  fires 
from  an  offshore  platform.  In  this  case,  the  offshore  platform  was  the  Navy’s  Stiletto  test 
ship  tied  up  on  the  Potomac  River. 
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UTACC  was  successful  in  both  of  the  scenarios  despite  hardware  limitations.  The 
issues  arose  from  the  UGV  and  UAV  platforms  themselves,  not  the  onboard  equipment. 
The  system  used  data  from  the  UGV  and  UAV  to  quickly  and  autonomously  build  an 
accurate  map  of  the  designated  operating  area,  satisfying  the  first  scenario’s  requirement. 
For  the  second  scenario,  the  system  effectively  built  on  the  first  scenario  map,  identified 
its  target,  and,  after  receiving  authorization,  requested  fire  and  engaged  the  target. 
Despite  the  limited  scope  of  the  LTA,  the  system  demonstrated  an  acceptable  level  of 
hardware  and  collaborative  capability.  The  system  autonomously  built  a  map  and 
identified  its  designated  targets  using  onboard  equipment  and  accomplished  all  of  its 
tasks  through  collaboration  between  the  ground  and  aerial  vehicles.  LTA  2  reinforced 
UTACC’ s  viability  as  a  system  and  its  potential  to  improve  Marines’  combat  capability. 

B.  AUTONOMY 

Little  research  explores  the  concept  of  UTACC;  however,  there  is  literature  that 
describes  automation  and  its  application  to  future  combat.  Shaker  and  Wise  (1988) 
explain  the  history  of  automation  and  robotics  going  back  to  World  War  I.  The  term 
autonomous,  as  used  in  robotics,  is  interpreted  in  a  number  of  ways  by  people  in  the 
industry.  Autonomy  can  range  from  direct  control  of  a  system  to  unmanned  systems 
executing  tasks  with  no  human  intervention  (Bruemmer,  Ferlis,  Huang,  Novak,  Schultz, 
&  Smith.,  2004).  Bruemmer  et  al.  (2004)  and  Glotzbach  (2004)  define  automation  and 
provide  guidelines  for  measuring  levels  of  autonomy.  UTACC  is  intended  to  develop  a 
semi-autonomous  system  that  enhances  decision-making  on  the  battlefield.  The  National 
Institute  of  Standards  and  Technology  defines  semi-autonomous  as  a  form  of  operation 
where  humans  and  machines  execute  missions  by  leveraging  various  levels  of  Human- 
Robot  Interaction  (HRI).  In  conjunction  with  the  previous  literary  sources,  Siegwart, 
Nourbakhsh,  and  Scaramuzza  (2011)  introduce  the  fundamentals  of  robotic  and  mobility 
autonomy. 

The  Role  of  Autonomy  in  Department  of  Defense  (DOD)  Systems  is  critical  to 
UTACC  development.  The  Role  of  Autonomy  in  DOD  Systems  defines  the  current  and 
future  role  of  autonomy  within  the  DOD.  The  document,  created  by  the  Department  of 
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Defense,  identifies  opportunities  and  challenges  in  the  integration  of  autonomous 
vehicles  in  the  military  (DOD,  2012).  The  DOD  recognizes  that  the  strength  of  unmanned 
aerial  vehicles  (UAV)  is  their  ability  to  provide  persistent  intelligence,  surveillance,  and 
reconnaissance  (ISR)  on  the  battlefield.  Increased  autonomy  enhances  these  capabilities 
and  the  safety  of  the  system.  Increased  autonomy  also  reduces  human  error  during  take¬ 
off,  flight,  and  landing,  increasing  the  safety  of  the  system.  The  safety  and  ISR  that 
increased  UAV  autonomy  provides  reduces  Marines’  cognitive  load.  UGVs  similarly 
reduce  Marines’  cognitive  load.  The  Role  of  Autonomy  in  DOD  Systems  describes  what 
UGVs  are,  their  benefits,  and  how  increased  autonomy  can  improve  them.  It  explains  that 
UGVs’  major  benefit  is  the  standoff  distance  they  provide  the  warfighter.  It  also  mentions 
the  need  for  a  UGV  capable  of  operating  and  making  decisions  in  accordance  with  the 
rules  of  engagement  (ROE)  and  a  commander’s  intent.  At  its  completion,  UTACC  will 
function  as  a  teammate  that  operates  in  accordance  with  Marine  Corps  doctrine  and  ROE. 

Gustavsson  and  Hieb  (2013)  address  the  challenges  associated  with  integrating 
autonomous  systems  in  the  military.  They  introduced  the  “Operations  Intent  and 
Effects  Model.”  Their  model  outlines  the  integration  of  future  Command  and  Control 
(C2)  systems  in  the  DOD  to  help  the  military  recognize  the  benefits  of  automation. 
Lin,  Beckey,  and  Abney  (2008)  add  to  this  discussion  by  listing  future  missions  and  task 
sets  for  robotic  systems  and  describing  possible  ethical  implications  of  robotic  systems’ 
use  in  war. 

Another  important  aspect  of  UTACC  is  Human-Robot  Interaction  (HRI).  Groom 
and  Nass  (2008,  p.  496)  suggest  that  the  following  question  guides  HRI  model 
development:  “which  human  inabilities  can  the  robot  perform,  and  what  organizational 
structure  best  supports  both  human  and  robots?”  They  note  that  future  robotic  teammates 
will  have  high  levels  of  autonomy  and  coordination  skills  to  assist  their  human 
counterparts,  but  that  system  functionality  may  be  limited  to  specific  environments  unless 
explicitly  designed  otherwise.  Integrating  an  HRI  framework  into  system  development 
creates  broad  system  application  and  increases  the  success  of  a  system  in  unpredictable 
environments  (Groom  &  Nass,  2008,  p.  483). 
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UT ACC’s  level  of  autonomy  is  significant  to  system  development.  As 
sophistication  increases,  human  operators  will  assume  a  supervisory  role  over  a  system 
instead  of  actively  controlling  it  (Chen  &  Barnes,  2014,  p.  30).  Maintaining  a  supervisory 
role  over  the  system  will  be  challenging  if  there  is  not  a  proper  system  interaction  or 
interface  to  support  the  human  operator  (Chen  &  Barnes,  2014,  p.23).  Incoming  sensor 
data  will  have  to  be  relayed  to  C2  workstations  and  displayed  in  a  comprehensible 
manner  to  be  useful  to  the  decision  maker  (Shattuck  &  Lewis  Miller,  2006,  p.  8).  Key 
leaders  must  be  supported  with  the  most  pertinent  information,  so  that  their  perceptions, 
understanding,  predictions,  and  decisions  meet  the  requirements  needed  to  accomplish 
the  system’s  goals  (Shattuck  &  Lewis  Miller,  2006,  p.  6).  Trafton  et  al.  (2006)  argue  that 
integrating  a  computational  cognitive  model  in  a  robotic  system  will  increase  its 
intelligence  and  create  superior  decision-maker  support  capability. 

Robot  and  Marine  must  exchange  information  to  accomplish  the  goals  of  the 
UTACC  system.  Gold  (2009)  outlines  four  areas  of  information  exchange:  robot  to 
human,  environment  to  robot,  human  to  robot,  robot  to  environment.  UTACC  success 
also  requires  that  information  flows  across  the  four  areas  in  addition  to  a  fifth,  robot  to 
robot.  The  UTACC  system  interacts  with  the  environment  through  UGV  and  UAV  on¬ 
board  sensors  while  simultaneously  transmitting  and  receiving  information  from  other 
robots  and  humans  via  communication  links. 

UTACC  must  be  included  within  the  Marine  Corps  Task  List  (MCTL)  to  evaluate 
capability  gaps  and  determine  efficiencies  (Rice,  Keim,  &  Chhabra,  2015,  p.  16).  The 
concepts  of  collaborative  autonomy  and  interoperability  do  not  exist  in  Marine  Corps 
doctrine.  However,  Marine  Corps  Doctrinal  Publication  (MCDP)-l  Warfighting  along 
with  MCDP-2  Intelligence,  MCDP-3  Expeditionary  Operations,  MCDP-4  Logistics, 
MCDP-5  Planning,  and  MCDP-6  Command  and  Control  are  the  fundamentals  of  Marine 
Corps  warfighting  and  should  be  the  basis  for  UTACC  concepts. 

C.  MOE/MOP 

In  March  2014,  the  Marine  Corps  published  Expeditionary  Force  21  outlining  the 
future  of  Marine  Corps  warfighting.  It  is  a  blueprint  for  the  Marine  Corps  capabilities  and 
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capacity  decision-making  processes  (USMC,  2014a,  p.  7).  Expeditionary  Force  21 
describes  a  modem  force  that  actively  integrates  innovation  and  emerging  technologies  to 
create  an  advantage  over  future  opponents  (USMC,  2014a,  p.  7).  UTACC  accomplishes 
that  vision  because  it  is  an  innovative  technological  concept  that  gives  decision  makers 
an  edge  over  their  enemies. 

UTACC  will  provide  commanders  with  an  advantage  by  integrating  all 
warfighting  functions  (intelligence,  maneuver,  fires,  logistics,  force  protection,  command 
and  control).  Initial  development,  however,  will  focus  on  the  intelligence  warfighting 
function.  Future  iterations  of  the  system  will  address  the  other  functions.  Development 
will  begin  by  analyzing  the  Marine  Corps  Tasks  (MCTs)  because  they  will  be  used  to 
build  measures  of  effectiveness  (MOE)  and  measures  of  performance  (MOP)  (Rice  et  al., 
2015,  p.  17).  The  Joint  Chief  of  Staffs  directorate  J-7  defines  and  explains  MOEs  and 
MOPs.  The  J-7  “is  responsible  for  the  six  functions  of  joint  force  development:  Doctrine, 
Education,  Concept  Development  &  Experimentation,  Training,  Exercises  and  Lessons 
Learned”  (Glossary,  n.d.).  MOEs  are  created  to  analyze  the  effects,  both  good  and  bad,  of 
a  system  on  operations  (JCS  J7,  2011,  p.  III-4).  They  prompt  an  organization  to  assess  its 
development  efforts  and  track  their  progress  toward  accomplishing  the  system’s  ultimate 
goals  (Rushing  &  Kirkpatrick,  2016).  MOPs  are  incremental  ties  that  link  system  tasks  to 
the  MCTL.  System  developers  use  MOPs  to  align  their  efforts  to  the  needs  of  the  service 
acquiring  the  system  (Rushing  et.  al,  2016).  For  UTACC,  the  Marine  Corps  Warfighting 
Lab  and  Naval  Postgraduate  School  are  the  developing  organizations  and  the  Marine 
Corps  is  the  acquiring  service.  System  developers  must  analyze  the  MCTs  and  the 
develop  MOEs  and  MOPs  to  ensure  incremental  UTACC  system  development  (Rice  et 
al.,  2015).  Incremental  development  requires  extensive  experimentation  that  should  be 
structured  pursuant  to  a  campaign  of  experimentation. 

D.  CAMPAIGN  OF  EXPERIMENTATION 

Experimentation  is  a  cornerstone  of  DOD’s  strategy  to  transform  the  current  force 
into  a  technologically  advanced,  net  centric  force.  UTACC  experimentation  will  create  a 
system  that  reinforces  network  capabilities,  improves  information  sharing,  and  increases 
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situational  awareness  and  mission  effectiveness  (Alberts,  Hayes,  Kirzl,  Leedom,  & 
Maxwell,  2005,  pp.  8-9).  Semi-autonomous  system  experimentation  enhances  the 
abilities  of  decision  makers  and  supports  the  physical,  information,  and  cognitive 
domains  of  net  centric  warfare  (NCW)  (Alberts  et  al.,  2005,  p.  10).  The  physical  domain 
of  NCW  refers  to  the  land,  air,  sea,  and  space  spheres  where  warfare  is  conducted  and 
combat  effectiveness  is  judged.  The  information  domain  is  where  information  is  created, 
shared,  manipulated,  and  where  most  command  and  control  is  communicated.  The 
cognitive  domain  is  in  the  mind  of  the  individual  warfighter.  The  cognitive  domain 
consists  of  the  tangibles  of  the  tactics,  techniques,  and  procedures  in  addition 
to  the  intangibles  of  leadership,  situational  awareness  and  more  (Alberts  et  al.,  2005, 
pp.  11-12).  The  DOD’s  command  and  control  research  programs  publication  Code 
of  Best  Practices:  Experimentation  discusses  the  simultaneous  relationships  that  take 
place  between  the  domains  of  NCW  to  ensure  mission  success.  Through  experimentation, 
UTACC  developers  will  leverage  the  relationships  between  domains  to  enhance 
mission  accomplishment. 

A  comprehensive  campaign  of  experimentation  will  guide  incremental  UTACC 
development.  A  framework  of  discovery,  hypothesis,  and  demonstration  experiments  in 
the  campaign  of  experimentation  accomplishes  incremental  system  development. 
Discovery  experiments  are  experiments  that  introduce  new  concepts,  technologies,  or 
systems  to  an  environment  for  analysis  (Alberts  et  al.,  2005,  p.  19).  They  determine  the 
military  viability  of  a  system,  potential  employment  methods,  and  conditions  for  or  the 
limits  of  the  systems’  use.  Discovery  experiments  can  be  conducted  in  a  number  of 
ways — ranging  from  simulators  to  actual  field  usage — to  facilitate  innovation  while 
reducing  cost.  Discovery  experimentation  is  a  precursor  to  hypothesis  experimentation 
because  hypotheses  are  created  during  this  phase  (Alberts  et  al.,  2005,  p.  20). 

Hypothesis  testing  is  the  next  phase  of  experimentation.  Hypothesis  testing 
determines  the  limiting  factors  of  the  system  and  tests  the  system  as  a  whole  (Alberts  et 
al.,  2005,  p.  22).  The  primary  goal  of  hypothesis  experimentation  is  to  gain  knowledge 
about  possible  variables  that  affect  the  functionality  of  the  system.  Initial  hypothesis  are 
reformed  through  subsequent  hypothesis  testing.  Because  hypotheses  are  tested  and 
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reformulated  through  experimentation,  system  developers  must  carefully  select  initial 
hypotheses  to  test  to  avoid  unnecessary  complexity  and  data  obfuscation  (Alberts  et  al., 
2005,  p.  22).  The  hypothesis  testing  phase  is  broken  into  two  parts,  a  preliminary  and 
refined  segment.  The  preliminary  phase  of  experimentation  addresses  the  original 
hypothesis  and  the  results  refine  future  hypotheses.  The  refined  hypothesis  is  then 
tested  to  both  ensure  the  viability  of  the  refined  hypothesis  and  to  ensure  that  the 
system  functions  under  a  variety  of  conditions  (Alberts  et  al.,  2005,  p.  22).  The 
results  identify  the  viability  of  a  system  for  military  use  and  refine  the  system  for 
demonstration  experimentation. 

Demonstration  experiments  show  the  acquiring  institution  that  the  system 
enhances  combat  effectiveness  and  mission  accomplishment  under  varying  conditions 
(Alberts  et  al.,  2005,  p.  23).  In  contrast  to  hypothesis  testing,  demonstration  experiments 
are  not  designed  to  create  knowledge.  Instead,  they  present  known  information  to 
individuals  who  are  not  familiar  with  the  system  or  data  that  has  been  created  during 
the  previous  experimental  phases.  System  developers  should  demonstrate  the 
system  under  conditions  specific  to  its  use  so  that  the  experiment  effectively  conveys 
the  viability  of  the  capabilities  of  the  system  (Alberts  et  al.,  2005,  p.  23).  System 
developers  will  identify  a  system’s  specific  conditions  during  the  extensive  discovery  and 
hypothesis  phases  of  the  campaign.  In  all  of  the  experiments,  data  collection  is  critical 
(Alberts  et  al.,  2006,  p.  22). 

For  a  campaign  plan  to  be  successful,  it  must  be  focused  and  identify  objectives. 
MOEs  and  MOPs  give  UTACC  developers  focus  and  objectives.  A  campaign  of 
experimentation  is  a  framework  that  identifies  key  variables  and  relationships  while 
effectively  reinforcing  MOEs  and  MOPs  (Alberts  et  al.,  2006,  p.  69).  In  addition,  a 
campaign  of  experimentation  creates  a  balance  between  variety  and  replication  (Alberts 
et  al.,  2006,  p.  64).  Experimentation  variety  allows  system  developers  to  identify  issues 
that  require  follow-on  experimentation  (Alberts  et  al.,  2006,  p.  65).  The  amount  of  variety 
in  a  campaign  of  experimentation  influences  how  robust  the  conclusions  from  the 
analysis  of  data  collected  are.  Replication  is  a  critical  principle  in  the  execution  of  a 
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campaign  of  experimentation  because  it  shows  that  the  results  of  the  experimentation  are 
not  unique  to  a  particular  set  of  conditions  (Alberts  et  al.,  2006,  p.  66). 

The  UTACC  campaign  of  experimentation  follows  the  principles  of  phase 
transition:  stage  acknowledgement,  nonlinear  progress,  resource  availability,  steering 
group  recognition,  and  a  broad  scope  of  experimentation  (Alberts  et  al.,  2006,  pp.  124- 
125).  It  is  necessary  for  the  system  development  team  to  acknowledge  the  transition  from 
one  stage  of  the  campaign  to  the  next  because  stage  recognition  creates  knowledge 
continuity  during  experimentation.  Continuity  provides  clarity  for  researchers  if 
experimentation  does  not  follow  a  linear  path.  Developers  need  to  be  aware  of  the 
possibility  of  nonlinear  development  progression  to  avoid  frustration  and  allow 
developers  to  adjust  to  unexpected  data  discoveries.  Senior  steering  groups  allocate 
resources.  Their  authority  should  be  recognized  throughout  the  experimentation  process. 
Lastly,  awareness  of  the  need  for  a  broad  scope  of  experimentation  reinforces  the 
system’s  versatility  (Alberts  et  al.,  2005). 

E.  COACTIVE  DESIGN 

Coactive  design  is  “a  fresh  design  perspective  built  on  interdependence,  a  more 
comprehensive  understanding  of  interdependence,  a  model  for  human-machine  systems, 
a  design  method,  and  a  new  tool  to  assist  with  system  design  and  analysis  called  the 
Interdependence  Analysis  (IA)  Table”  (Zach,  2016,  p.  16).  Captain  Matt  Zach’s  thesis, 
“Unmanned  Tactical  Autonomous  Control  and  Collaboration  Coactive  Design,” 
introduced  coactive  design  to  the  UTACC  program.  Coactive  design  is  critical  for  the 
UTACC  program  because  it  defines  where  machine  automation  can  be  useful,  identifies 
key  machine  human  interactions,  and  improves  the  likelihood  that  unmanned  systems 
will  function  as  team  members.  Coactive  design  also  describes  how  the  close  relationship 
between  man  and  machine  can  be  accomplished  through  the  execution  of  a  set  of  shared 
goals.  Using  interdependence  analysis  tables,  designers  are  able  to  better  understand  the 
human-machine  relationship  and  can  therefore  gain  valuable  insight  into  the  coordination 
needed  to  accomplish  different  goals  (Johnson,  2014,  p.  46).  There  are  three  specific 
realms  where  interdependence  exists:  observability,  predictability,  and  directability. 
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Coactive  design’s  design  flexibility  and  relationship  understanding  ensures  that  humans 
and  machines  operate  together  to  accomplish  goals  while  broadening  the  machines’ 
capability  to  accomplish  its  immediate  task  and  overall  goals. 

F.  MODEL  DRIVEN  SOFTWARE  DEVELOPMENT 

Model  driven  software  design  (MDSD)  is  another  design  methodology  that 
supports  system  development  and  flexibility.  MDSD  aims  to  increase  development  speed, 
improve  software  quality,  improve  software  maintenance,  increase  reusability,  manage 
system  complexity,  and  increase  interoperability  (Stahl,  Volter,  Bettin,  Haase,  Helsen,  & 
Czarnecki,  2006,  p.  13).  This  methodology  uses  software  models.  A  model  is  “an  abstract 
representation  of  a  system’s  structure,  function  or  behavior”  (Stahl  et  al.,  2006,  p.  18). 
Models  are  used  to  document  the  structure  of  software  for  complex  development  projects 
(Stahl  et  al.,  2006,  p.  18).  The  models  are  related  to  the  system  through  mapping  and 
should  include  information  about  the  system,  rules  for  the  system,  and  the  definitions  for 
terminology  used  in  the  system  (Siegel,  2014,  p.  5).  The  modeling  process  itself  creates 
an  architecture.  The  architecture  defines  the  system  or  system  of  systems.  UTACC  is  a 
system  of  systems  (Siegel,  2014,  p.  6).  The  UTACC  system  uses  several  complex 
software  suites  working  collaboratively  to  complete  a  specific  task  and/or  series  of  tasks. 
The  models  creating  the  architecture  will  provide  the  software  developers  the  “exact 
meaning  of  program  code”  for  the  finalized  UTACC  product  improving  the  software 
quality  and  development  speed  (Stahl  et  al.,  2006,  p.  14). 

Creating  an  architecture  through  the  modeling  process  falls  within  the  purview  of 
Architecture-Centric  MDSD  (AC-MDSD).  AC-MDSD  is  an  approach  developers  can  use 
to  effectively  organize  complicated  software  structures.  AC-MDSD  is  structured  to  assist 
the  developer  in  avoided  coding  errors  by  increasing  the  quality,  efficiency,  and 
reusability  of  software  (Stahl  et  al.,  2006,  p.  21).  AC-MDSD  uses  the  architecture 
developed  through  modeling  as  a  blueprint  that  software  development  teams  can  use  as  a 
foundation  to  create  an  application  (Stahl  et  al.,  2006,  p.  22).  The  more  detailed  the 
architecture,  the  more  robust  a  blueprint  is  created.  The  more  robust  the  blueprint,  the 
more  efficient  software  developers  are  because  they  can  copy  source  coding  rather  than 
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create  new  coding  for  a  function  (Stahl  et  al.,  2006,  p.  22).  The  increased  efficiency, 
quality,  and  reusability  of  software  generated  by  AC-MDSD  method  benefits  both  short¬ 
term  and  long-term  UTACC  development.  Software  developers  working  on  future 
iterations  of  the  system  can  use  or  modify  previously  generated  architectures  and  coding 
to  meet  future  requirements.  UTACC  system  development  that  employs  AC-MDSD 
techniques  will  create  an  enduring  system  that  meets  the  warfighting  needs  of  the  Marine 
Corps. 


G.  SECTION  CONCLUSION 

UTACC  is  a  unique  program  within  the  Marine  Corps  because  it  transitions 
unmanned  systems  from  Marines’  tools  to  their  teammates.  To  accomplish  this,  the 
Marine  Corps  is  increasing  human-robot  interaction  by  using  and  advancing  the 
autonomy  of  unmanned  systems.  The  literature  review  is  a  source  for  information  about 
UTACC  and  how  autonomy  plays  a  role  ensuring  the  success  of  the  system.  The  next 
chapters  of  this  thesis  provide  greater  detail  regarding  the  campaign  of  experimentation, 
its  parts,  and  different  design  methods.  This  thesis  includes  templates  for  how  to  plan  and 
organize  future  experimentation  so  that  UTACC  can  evolve  from  a  concept  into  an 
operational  system. 
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III.  RESEARCH  METHODOLOGY 


A.  SYSTEMS  ENGINEERING  APPROACH 

This  thesis  utilizes  the  research  methodology  known  as  the  systems  engineering 
approach.  All  prior  UTACC  theses  have  described  and  applied  this  methodology.  This 
methodology  originates  from  Benjamin  S.  Blanchard’s  Systems  Engineering 
Management  (4th  edition)  textbook,  where  Blanchard  outlines  the  systems  engineering 
approach.  Blanchard  defines  a  system  as  “a  construct  or  collection  of  different  elements 
that  together  produce  results  not  obtainable  by  the  elements  alone.”  Rice  et  al.  explain 
in  their  thesis,  “Unmanned  Tactical  Autonomous  Control  and  Collaboration  Concept 
of  Operations,”  that  UTACC  will  ideally  function  as  a  system  of  systems,  collaborating 
to  enhance  mission  accomplishment  (Rice  et  al.,  2015,  p.  20).  The  campaign  of 
experimentation  for  UTACC  meets  the  systems  engineering  step  of  detail  design  and 
development.  This  thesis  uses  the  concept  of  operations  (CONOP)  to  outline  the 
campaign  of  experimentation. 

B.  CAMPAIGN  OF  EXPERIMENTATION 

The  fundamental  anatomy  of  a  campaign  of  experimentation  consists  of  a 
centralized  focus,  a  set  of  objectives  to  gauge  the  success  of  the  campaign,  and  variety 
and  replication  in  how  experiments  are  staged  and  hypothesis  are  refined  (Alberts  et  al., 
2006,  p.  69).  Each  objective  has  a  set  of  measures  associated  with  it  to  help  analyze  the 
effects  of  specific  capabilities  being  tested  and  tie  them  back  to  essential  Marine  Corps 
tasks  (Alberts  et  al.,  2006,  p.  69).  Variety  and  replication  are  the  guiding  principles  used 
for  planning  experimentation  for  the  campaign  of  experimentation.  They  ensure  that  a 
successful  system  is  developed  in  a  comprehensive  and  incremental  way  (Alberts  et  al., 
2006,  p.  64). 

UTACC  will  ultimately  be  a  system  of  systems  that  reduces  the  cognitive  load  felt 
by  Marines  in  combat,  thereby  enhancing  Marines’  ability  to  accomplish  missions.  This 
will  be  accomplished  by  integrating  autonomous  robots  into  Marine  Corps  units  (Rice  et 
al.,  2015,  p.  20).  This  is  the  ultimate  goal  of  UTACC  and  serves  as  a  centralized  focus 
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over  the  course  of  the  campaign  of  experimentation.  The  next  step  is  to  identify  the 
objectives  that  must  be  accomplished  to  make  UTACC  goals  a  reality.  The  broadest 
concrete  objective  of  UTACC  is  to  develop  a  system  prototype  and  evaluate  its 
capabilities  (Alberts  et  al.,  2006,  p.  70).  To  create  the  system  prototype,  system 
developers  assess  system  efficacy  by  comparing  experiment  data  against  entrance  and 
exit  criteria,  otherwise  known  as  MOEs  and  MOPs.  Preliminary  MOEs  and  MOPs  for 
UTACC  are  provided  in  the  fourth  thesis  of  the  series,  “UTACC  Measures  of 
Performance  and  Measures  of  Effectiveness”  (2016)  and  they  are  reproduced  here  in 
Appendix  C. 

The  previous  chapter  states  that  the  DOD’s  strategy  for  transforming  the  military 
into  a  net  centric,  technologically  advanced  force  relies  on  extensive  experimentation. 
Experimentation  for  UTACC  will  occur  during  both  limited  technical  assessments 
(LTAs)  and  limited  objective  experiments  (LOEs).  Although  the  current  thesis  describes 
how  system  developers  can  structure  LTAs,  developers  may  use  limited  objective 
experiments  (LOE)  to  advance  UTACC  knowledge  and  development. 

To  thoroughly  understand  the  campaign  of  experimentation,  one  must  look  at  its 
most  fundamental  element,  the  experiment  itself.  The  Command  and  Control  Research 
Program’s  (CCRP)  book  Experimentation  (2005)  explains  that  there  are  three  types 
of  experimentation:  discovery,  hypothesis  and  demonstration  experiments.  Figure  1 
illustrates  the  process  of  experimentation  and  what  could  result  from  it. 
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Figure  1.  Process  of  Experimentation.  Source:  Alberts  et  al.  (2005,  p.  26). 

Discovery  experiments  are  experiments  where  system  developers  introduce  new 
concepts,  technologies,  or  systems  to  an  environment  where  their  impact  can  be  recorded 
and  analyzed  (Alberts  et  al.,  2005,  p.  19).  Military  technologists  have  traditionally  relied 
on  discovery  experiments  to  determine  the  utility  of  a  technology  before  giving  it  to  end 
users  to  create  a  concept  of  operations  (Alberts  et  al.,  2005,  p.  20).  The  UTACC  system  is 
different  because  a  preliminary  concept  of  operations  has  been  formulated  and 
technology  is  currently  being  adapted  to  fit  that  role.  Although  a  preliminary  concept  of 
operations  was  created,  experimentation  for  UTACC  fits  within  the  parameters  of 
discovery  experimentation  because  the  experiments  use  mature  technologies  to  develop 
new  applications  for  those  technologies  and  potentially  refine  the  use  of  those 
technologies.  Ultimately,  UTACC  discovery  experimentation  will  determine  whether  the 
system  is  militarily  viable,  how  the  system  can  be  used,  and  what  conditions  extend  or 
limit  the  systems’  use  (Alberts  et  al.,  2005,  p.  20). 
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After  discovery  experimentation,  hypothesis  experimentation  investigates 
different  variables  that  impact  the  system.  Hypothesis  testing  is  done  in  two  phases,  a 
preliminary  phase  and  a  refinement  phase.  Additionally,  it  requires  a  number  of 
experiments  to  fully  test  the  hypothesis  (Alberts  et  al.,  2005,  p.  22).  The  preliminary 
phase  of  the  experimentation  addresses  the  hypothesis  selected,  with  the  results  informing 
the  hypothesis  refining  process.  The  newly  refined  hypothesis  is  then  tested  under  a 
variety  of  conditions  to  verify  the  system’s  efficacy  (Alberts  et  al.,  2005,  p.  22).  UTACC 
hypotheses  are  based  on  critical  operational  issues  (COI)  identified  to  steer  the  conduct  of 
experimentation.  A  critical  operational  issue  is  “key  operational  effectiveness  or 
suitability  issues  that  must  be  examined  in  operational  test  and  evaluation  to  determine 
the  system’s  capability  to  perform  its  mission”  (Glossary,  n.d.).  This  thesis  proposes 
seven  COIs:  1)  Will  the  system  reduce  the  cognitive  load  of  the  team?  2)  Will  the  system 
render  enhanced  3-dimensional  reconnaissance  products?  3)  Will  the  system  increase  the 
safety  of  the  team?  4)  Will  the  system  enhance  identification  and  engagement  of  targets? 
5)  Does  the  system  operate  in  accordance  with  Marine  Corps  doctrine?  6)  To  what  extent 
does  the  digital  plan  provide  context  to  the  machines  as  well  as  the  Marines?  7)  Does  the 
system  demonstrate  flexibility  to  changes  in  the  environment/plan? 

The  final  form  of  experimentation  identified  by  the  CCRP  is  demonstration 
experimentation.  Demonstration  experimentation  will  show  that  UTACC  enhances 
combat  effectiveness  and  mission  accomplishment  under  a  variety  of  conditions 
described  in  Chapter  IV  (Alberts  et  al.,  2005,  p.  23).  Conducting  the  experiments  in 
settings  that  the  system  will  be  used  in  will  properly  showcase  its  capabilities  (Alberts  et 
al.,  2005,  p.  23).  These  conditions  are  identified  based  on  the  nature  of  the  previous  two 
experimental  phases  (Alberts  et  al.,  2005,  p.  23).  UTACC  experimentation  is  designed  in 
increments.  As  each  increment  builds  on  the  last,  the  conditions  the  system  is  tested  in 
evolve  in  complexity.  The  conditions  or  scenario  remain  the  same  for  each  stage  of 
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experimentation,  only  the  technological  capabilities  of  the  system  change/evolve.  Figure 
2  displays  the  nature  of  a  campaign  of  experimentation  and  demonstrates  how  it  follows 
an  incremental  incline  as  it  progresses.  As  experimentation  continues,  the  complexity  of 
the  system  increases,  refining  the  conditions  for  experimentation  and  advancing  the 
knowledge  of  system  capabilities  (Alberts  et  al.,  2005,  p.  49). 


Figure  2.  Nature  of  a  Campaign  of  Experimentation. 

Source:  Alberts  et  al.  (2005,  p.  49). 

Following  this  type  of  trajectory,  the  proposed  campaign  of  experimentation  will 
drive  the  progress  of  the  UTACC  system  from  concept  to  reality. 

C.  COACTIVE  DESIGN 

Coactive  design  is  a  design  methodology  that  focuses  on  human-machine 
interaction  which  will  be  useful  to  UTACC  developers.  It  is  “a  fresh  design  perspective 
built  on  interdependence,  a  more  comprehensive  understanding  of  interdependence,  a 
model  for  human-machine  systems,  a  design  method,  and  a  new  tool  to  assist  with  system 
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design  and  analysis  called  the  Interdependence  Analysis  (IA)  Table”  (Zach,  2016,  p.  16). 
Captain  Matt  Zach’s  thesis,  “Unmanned  Tactical  Autonomous  Control  and  Collaboration 
Coactive  Design,”  applies  the  design  process  to  UTACC  using  the  Marine  Corps 
planning  process  BAMCIS  (Begin  the  planning,  Arrange  reconnaissance,  Make 
reconnaissance,  Complete  the  plan,  Issue  the  order,  and  Supervise).  Appendix  A  models 
how  BAMCIS  applies  to  UTACC.  This  section  gives  an  overview  of  the  coactive  design 
process  and  explains  how  its  use  throughout  the  execution  of  the  campaign  of 
experimentation  will  greatly  enhance  UTACC  development. 

Dr.  Matt  Johnson  of  the  Florida  Institute  of  Human  and  Machine  Cognition 
(IHMC)  believes  that  the  coactive  design  process  is  superior  to  others  because  it  focuses 
on  the  interdependent  relationship  between  human  and  machine  (Zach,  2016,  p.  17).  The 
coactive  design  method  captures  the  concepts  of  coordination,  cooperation,  and 
collaboration  and  conveys  them  in  a  requirements  based  format.  The  method  consists  of 
three  processes:  1)  the  identification  process,  2)  selection  and  implementation,  and  3)  the 
evaluation  of  change  processes  (Zach,  2016,  p.  24).  Each  process  is  then  broken  down 
into  a  series  of  subordinate  processes.  The  inputs  and  outputs  required  for  those  sub¬ 
processes  are  defined.  Figure  3  provides  a  visual  representation  of  the  coactive  design 
process. 

Zach  (2016)  modified  the  original  coactive  design  task  analysis  worksheets 
into  UTACC  interdependence  analysis  (IA)  tables.  Appendix  B  is  an  example  of  an  IA 
table.  The  modified  tables  address  the  overarching  tenets  identified  within  the  CONOPS 
while  also  identifying  and  addressing  shortfalls  that  were  not  conceived  by  Rice  et  al. 
(2015)  (Zach,  2016,  p.  3).  These  tables  assist  developers  in  identifying  the  different 
variables  associated  with  interdependence,  tasks  to  be  completed,  and  the  relationship 
between  the  two. 
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Identification  pnxm 

"'p'*  Traditional  talk  analysis. 

The  composition  of  the  team  and  knowledge  of  then  capabilities. 

Anticipated  situation  alternatives. 

i  — 

process  Entity  required  capacities  leg.  knowledge,  information,  skills,  and  abilities)  lor  each  task. 

- - — - 

Enumerate  viable  team  ro*e  alternatives. 

1  ~ 

Assess  capacity  to  perform  and  capacity  to  support. 

V  ~ 

Identify  the  potential  interdependence  relationships  In  the  |olnt  activity 
Look  for  required  relationships  based  on  lack  of  capacity. 

Look  for  required  relationships  based  on  capacity  that  is  <  100  percent  reliable 

Look  for  opportunistic  relationships  based  on  capacity  overlap 


Determine  the  specific  OPD  requirements  of  these  relationships. 

Identify  what  each  party  needs  to  make  observable  to  the  others. 

*  ~ 
Identify  behaviors  that  need  to  be  predictable  by  others. 

—  V 

Identify  the  ways  parties  need  to  direct  each  other. 

'  t  . 

ounwi  f  description  of  a  set  of  interdependence  relationships  that  should  be  supported 
in  the  implemented  system  and  the  associated  OPD  requirements  of  each. 


lion  and  implementation  process 

>n(>ut  A  description  of  a  set  of  interdependence  relationships  that  should  be  supported 

in  the  implemented  system  and  the  associated  OPD  requirements  of  each. 

T 

process  fot  each  relationship  decide  on  appropriate  mechanrsms  to  support  the  requirements 

Identify  mechanisms  that  ensure  sufficient  support  of  OPO  requirements 

P=  _  * 

Select  and  implement  the  mechanism. 

* 

A  implementation  of  a  particular  mechanism  designed  to  support  a  particular 
Interdependence  relationship  and  Its  OPO  requirements 


I  Evaluation  of  change  process 

“’P*  I  A  implementation  of  a  particular  mechanism  designed  to  support  a  particular 

interdependence  relationship  and  its  OPD  requirements 


Does  this  imptemenution  affect  the  required  OPO  In  any  other  relationships? 

1  no 

Does  this  imptemenution  add.  remove,  or  alter  any  interdependence  relationships’ 

1  no 

Possible  solution.  Evaluate  using  traditional  human  factors  and  performance  analysis 


Figure  3.  Coactive  Design  Method.  Source:  Johnson  (2014). 
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Coactive  design  is  a  unique  combination  of  the  waterfall  and  spiral  design 
models.  The  waterfall  attributes  of  the  process  make  it  easier  to  follow  and  execute  while 
the  spiral  model  attributes  facilitate  adaptation  throughout  the  design  process  (Satzinger, 
Jackson,  &  Burd,  2012,  pp.  228,  230).  The  simplicity  and  flexibility  of  the  merged 
design  methods  give  the  UTACC  development  team  an  advantage  because  they  can 
quickly  identify  interdependence  variables  and  either  add,  subtract,  or  modify  variables 
throughout  the  process.  The  method  also  helps  developers  generate  what  Dr.  Johnson 
(2014)  states  is  a  better  understanding  of  the  human-machine  relationship  because 
developers  must  identify  variables  that  impact  that  relationship.  Therefore,  developers 
gain  valuable  insight  into  the  coordination  needed  to  accomplish  different  goals.  With  the 
insight  provided  by  the  coactive  design  process,  developers  will  be  able  to  create  a 
system  that  can  better  function  as  an  autonomous  robotic  team  member  rather  than  a  tool 
for  Marines  to  operate  on  the  battlefield. 

D.  MODEL  DRIVEN  SOFTWARE  DEVELOPMENT 

The  UTACC  campaign  of  experimentation  described  in  this  thesis  also  includes 
model  driven  software  development  as  a  design  methodology.  DOD  publication  5000.02 
Operation  of  the  Defense  Acquisition  System  outlines  the  requirements  for  software 
development  for  DOD  programs.  It  states  that  the  development  of  software  should  be 
executed  in  a  comprehensive,  incremental,  and  efficient  way  in  order  to  reduce  cost 
and  schedule  overruns  (DOD,  2015,  p.  10).  Model  driven  software  development  best  fits 
the  DOD  standard  for  software  development.  The  goals  of  MDSD  are  to  increase 
development  speed,  improve  the  quality  of  the  software  created,  improve  software 
maintenance,  increase  reusability,  manage  the  complexity  of  system,  and  increase 
interoperability  (Stahl  et  al.,  2006,  p.13-14).  To  accomplish  these  goals,  software 
developers  utilize  models  to  individually  represent  the  system’s  characteristics,  like  its 
function  and  structure  (Stahl  et  al.,  2006,  p.18).  These  models  can  be  completed  in  a 
number  of  program  languages;  however,  the  most  commonly  used  is  the  unified 
modeling  language  (UML)  2.5.  As  a  result,  this  thesis  recommends  using  UML  2.5. 
System  developers  create  models  by  mapping  the  system,  which  includes  defining  system 

rules  and  defining  the  terms  used  in  the  system  (Siegel,  2014,  p.  5). 
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The  most  important  step  of  the  MDSD  process  is  meta-model  creation.  The  meta¬ 
model  provides  a  description  of  the  models’  structure  and  defines  the  modeling  language 
(Stahl  et  al.,  2006,  p.  85).  Classes  make  up  the  models’  structure  and  are  a  category  that 
describes  an  object  or  thing.  Each  class  contains  common  attributes  or  specific 
descriptors  (Satzinger,  2012,  p.  96).  For  UTACC,  the  classes  for  the  meta-model  would 
be  derived  from  BAMCIS  with  each  planning  phase  being  its  own  class  (Satzinger,  2012, 
p.  101).  For  example,  begin  planning  is  its  own  class,  with  attributes  of  system 
initialization  and  mission  parameters  (Rice  et  al.,  2015,  p.  39). 

After  system  developers  create  a  meta-model,  they  should  create  an  UMF  profile 
to  define  the  structure  of  the  model  and  model  constraints  (Stahl  et  al.,  2006,  p.  19).  With 
the  meta-model  complete,  the  models  themselves  are  created.  Fike  the  meta-model,  the 
models  are  made  up  of  classes  with  their  own  attributes,  but  they  are  created  using  the 
structure  and  language  defined  by  the  meta-model.  With  UTACC,  the  meta-model 
describes  the  use  of  BAMCIS  for  the  creation  of  the  models.  The  individual  models  will 
address  the  planning  phases.  The  resultant  models  make  up  an  architecture  that  serves  as 
an  overarching  definition  of  the  systems,  or  for  UTACC  a  system  of  systems  (Siegel, 
2014,  p.  6). 

Architecture-centric  MDSD  is  integrated  into  the  UTACC  software  development 
process.  AC-MDSD  is  structured  to  assist  the  developer  in  avoided  coding  errors  by 
increasing  the  quality,  efficiency,  and  reusability  of  software  (Stahl  et  al.,  2006,  p.  21). 
The  ability  to  attain  these  goals  falls  within  DOD  5000.02  requirements  for  software 
development.  By  taking  the  time  up  front  to  create  a  thorough  architecture,  efficiency  is 
increased.  The  architecture  facilitates  the  creation  of  source  code  developers  can  use  as  a 
blueprint.  The  blueprints  of  source  coding  that  are  created  and  can  be  used,  reused,  or 
modified  are  called  generative  software  architectures.  As  different  UTACC  software 
teams  begin  to  build  code  that  satisfies  different  functions  within  UTACC  software 
architecture,  other  teams  are  able  to  copy  that  source  coding  rather  than  create  it  anew. 
The  use  of  generative  software  architecture  increases  the  efficiency  of  the  development 
process,  the  interoperability  of  UTACC  software,  and  the  ability  to  easily  modify  the 
system  in  more  sophisticated  phases  of  development  (Stahl  et  al.,  2006,  p.  22). 
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Generative  software  architecture  is  a  key  component  of  AC-MDSD  because  it  facilitates 
the  modular  development  of  an  application. 

Before  software  development  begins,  developers  must  read  and  understand  the 
concepts  introduced  in  the  CONOPs.  The  structure  of  the  model  as  well  as  the  behaviors 
intended  for  the  system  to  exhibit  are  built  within  the  BAMCIS  planning  process  and  are 
described  in  the  CONOPs.  The  architecture  and  models  describe  the  system,  and  generate 
coding  that  will  be  used  throughout  software  development.  The  ability  to  generate  coding 
in  the  design  process  that  can  later  be  reused  and/or  modified  is  a  tremendous  strength  of 
MDSD  and  helps  to  further  increase  speed,  quality,  and  efficiency,  while  reducing  cost. 

E.  SECTION  CONCLUSION 

The  systems  engineering  approach  is  the  research  methodology  most  applicable  to 
UTACC  and  for  this  thesis.  The  campaign  of  experimentation  falls  within  the  detailed 
design  and  development  stage  of  the  systems  engineering  model.  The  campaign  starts 
with  discovery  experimentation,  evolves  to  hypothesis  experimentation,  and  finally 
extends  to  demonstration  experimentation.  Discovery  experiments  refine  the  uses  of 
technologies  being  adapted  for  UTACC.  Demonstration  experiments  will  display  the 
system’s  developing  capabilities,  interoperability,  and  the  interdependence  with  its 
human  counter-parts.  System  developers  will  conduct  experiments  during  LTAs  when  all 
development  parties  are  present. 

A  campaign  of  experimentation  can  incorporate  different  methodologies  to 
produce  the  best  results.  The  methodology  to  facilitate  the  quality,  speed,  and  efficiency 
of  interdependence  development,  as  required  by  DOD  5000.02,  for  UTACC  is  coactive 
design.  Using  interdependence  analysis  tables,  developers  can  identify  the  different 
UTACC  specific  variables  that  will  move  the  technology  from  tool  to  teammate.  Another 
method  that  improves  quality,  speed,  and  efficiency  of  software  development  is  model 
driven  software  development.  Utilizing  the  UTACC  CONOPs,  software  developers  will 
be  able  to  thoroughly  outline  and  build  models  and  architectures  that  are  in  line  with 
Marine  Corps  doctrine.  Being  able  to  create  blueprints  with  associated  coding  that  can  be 
used  by  a  number  of  development  teams  improves  the  efficiency  of  the  development 
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process,  saves  money,  and  enhances  the  quality  of  the  software.  Ultimately,  the  campaign 
of  experimentation  and  recommended  methodologies  take  UTACC  from  concept  to 
functional,  mission  enhancing  reality. 
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IV.  UTACC  CAMPAIGN  OF  EXPERIMENTATION 


UTACC  is  a  unique  combination  of  software  and  hardware  that  functions 
autonomously  while  collaborating  with  Marines.  The  ability  of  the  system  to  reduce 
Marines’  cognitive  load  is  critical  to  mission  and  system  success.  A  campaign  of 
experimentation  ensures  UTACC  meets  these  goals.  The  campaign  of  experimentation 
incrementally  balances  software  and  hardware  capabilities  in  accordance  with  the 
CCRP’s  principles  of  variety  and  replication.  The  campaign  is  structured  around  limited 
technical  assessments  because  they  serve  as  the  primary  setting  for  UTACC 
experimentation . 

A.  ORGANIZATION 

Appendix  D  is  a  Microsoft  Project  Gantt  chart  outlining  the  proposed  timeline, 
iterations,  and  focus  of  future  LTAs.  Because  funding  for  UTACC  is  currently 
guaranteed  until  2019,  the  chart  begins  where  LTA  two  finished  and  runs  through  2019. 
The  chart  is  organized  by  LTAs  and  displays  the  primary  focuses,  design  methodology, 
acceptance  testing,  follow-on  acceptance  testing,  and  correction  times  for  each  LTA.  Two 
LTAs  are  scheduled  to  take  place  each  year.  This  allows  a  six  month  period  for 
developers  to  conduct  their  own  experimentation/testing  and  address  any  issues  that  arise. 
Figure  4  is  an  example  timeline  for  LTAs  four  and  five. 


RacognlUvM 


p  LTA  Preparation  Begins  ©  Status  Update  (2  wee*  <><**) 

©  Acceptance  testing  *5  business  days 
©  Follow-on  testing  *  S  business  days 
A  Capability  Experimentation 

f  Limited  Technology  Assessment  (LTA)  •  5  business  days 


Figure  4.  LTA  Timeline 
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Each  LTA  advances  the  ability  of  UTACC  to  function  as  a  teammate  by  building 
on  the  accomplishments  of  the  previous  LTAs.  LTA  two  took  place  indoors  and 
demonstrated  the  ability  of  the  system  to  autonomously  build  a  3D  map,  search  for  a 
designated  target,  and  engage  the  target.  The  environment  the  experiment  was  conducted 
within  was  a  simulated  urban  environment.  The  target  was  identified  using  facial 
recognition  technology,  and  the  target  was  engaged  by  an  offshore  platform.  The 
unmanned  air  and  ground  vehicles  both  executed  3D  mapping  and  facial  recognition,  that 
the  air  and  ground  vehicles  also  worked  collaboratively  to  create  a  robust  and  accurate 
picture  of  both  the  physical  and  human  terrain.  The  focus  of  LTA  three  is  to  transfer 
these  capabilities  from  an  indoor  environment  to  an  outdoor  one.  During  LTA  three, 
UTACC  will  be  given  the  additional  tasks  of  reacting  to  a  new  environmental  variable 
and  building  a  plan  for  the  approach  to  a  designated  target  or  location.  The  system  must 
accomplish  the  measures  outlined  in  the  MOEs  and  MOPs  for  this  stage  of 
experimentation  before  the  system  qualifies  for  the  next  stage.  Coactive  design  and 
model  driven  software  development  are  incorporated  into  LTA  three  and  all  follow-on 
LTAs  as  parallel  efforts. 

LTA  four  experiments  with  voice  command  recognition  capabilities  to  reduce 
Marine’s  cognitive  load  when  operating  the  system.  Voice  recognition  allows  the  team 
leader  to  communicate  with  the  system  like  he  or  she  would  with  any  member  of  their 
team,  reducing  and  possibly  eliminating  the  need  for  the  Marine  to  interact  with  the 
system  through  a  physical  interface  like  a  tablet.  Voice  recognition  experimentation  may 
take  place  in  parallel  with  or  after  the  measures  accomplished  in  previous  LTAs  are 
repeated.  It  should  be  understood  that  the  voice  recognition  referenced  here  does  not 
mean  full  dialogue  between  the  Marine  and  the  machine.  This  stage  of  experimentation 
tests  the  system’s  ability  to  receive  basic,  directional  voice  commands.  The  commands 
“forward,”  “reverse,”  “left,”  “right,”  and  “stop”  are  the  suggested  goals.  Anything  more 
complex  than  this  can  be  reached  in  future  experimentation/development. 

LTA  five  develops  Marine-machine  communication  by  testing  the  system’s 
ability  to  auto-follow  and  understand  hand  and  arm  signals.  Marines  on  patrol  maintain 
formation  and  communicate  non- verbally  with  one  another.  During  early- stage 
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experimentation,  the  system  should  be  able  to  recognize  and  maintain  formation  while 
following  a  designated  member  of  the  team.  The  system’s  autonomous-follow  ability 
allows  Marine  teammates  to  maintain  situational  awareness  of  their  surroundings  without 
distraction  from  the  machine.  Hand  and  arm  signal  recognition  similarly  increases 
Marines’  situational  awareness,  thereby  enhancing  combat  capability.  The  UTACC 
CONOPS  describe  a  scenario  where  a  small  reconnaissance  team  is  inserted  into  a  region. 
As  the  team  moves  through  its  area  of  operations,  nonverbal  communication  maintains 
the  covertness  and  safety  of  the  team  and  mission.  Hand  and  arm  signals  can  be  used  to 
communicate  everything  from  a  moment’s  pause  to  a  change  in  patrol  formation.  At  this 
stage,  the  hand  and  arm  signals  that  LTA  four  tests  are  basic  directional  commands 
“forward,”  “reverse,”  “left,”  “right,”  and  “stop.”  This  establishes  a  baseline  that  can  be 
built  on  in  future  iterations  of  the  system.  Multiple  means  of  communication  and 
confidence  in  the  functionality  of  the  system  facilitates  Marines’  intuitive  use  of  the 
machine. 

LTA  six  tests  the  planning  and  maintenance  capabilities  of  the  system,  with  a 
focus  on  threat  analysis  and  self-diagnostics.  Early  stage  planning  ability  testing  begins  in 
LTA  three.  In  LTA  three,  the  system  must  provide  a  basic  plan  for  an  approach  to  a 
designated  target.  The  planning  at  that  stage  does  not  take  into  account  potential  threats 
to  the  team;  it  only  recommends  the  most  straightforward  approach  available.  LTA  six 
will  test  if  the  system  can  identify  potential  threats  to  the  team  and  incorporate  that 
information  into  a  plan.  The  plan  is  presented  to  the  team  leader  as  a  recommendation 
subject  to  acceptance,  rejection,  or  modification.  Because  this  is  early  stage 
experimentation,  the  expectation  for  threat  analysis  must  be  simple.  At  this  stage,  the 
system  should  be  able  to  identify  a  basic  linear  danger  area,  like  a  road.  The  second  focus 
of  LTA  six  is  self-diagnostic  ability.  Like  any  team  member  capable  of  communicating 
current  physical  condition  information  to  the  team  leader,  UTACC  must  be  able  to 
express  its  condition  to  the  team  leader.  Information  regarding  the  condition  of  the 
system  is  not  only  critical  for  maintenance  reasons,  but  it  also  plays  a  role  in  the  decision¬ 
making  process  of  the  team-leader.  If  the  system  has  degraded  for  any  reason,  knowing 
this  will  allow  the  team  leader  to  decide  whether  or  not  to  use  the  system  and  its 
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remaining  functional  capabilities.  Self-diagnostic  capabilities  means  maintainers  can 
quickly  identify  and  repair  issues,  thereby  saving  time  and  money  during  maintenance 

Lastly,  all  LTAs  should  include  acceptance  and  follow-on  testing.  During  LTA 
two  it  became  clear  that  experimentation  and  technology  tests  had  not  occurred  prior  to 
the  LTAs  execution.  This  resulted  in  slower  progress.  During  acceptance  and  follow-on 
testing,  project  managers  and  the  Marine  Corps  Warfighting  Laboratory  assess  the 
progress  of  development  teams  and  technology,  facilitate  coordination  between  team 
efforts,  and  identify  capability  gaps  prior  to  the  LTA.  The  current  campaign  of 
experimentation  schedules  acceptance  testing  60  days  prior  to  the  LTA  and  follow-on 
testing  30  days  prior  to  the  LTA.  Both  acceptance  and  follow-on  testing  are  scheduled  to 
occur  over  a  five-day  period.  Correction  time  occurs  after  acceptance  testing  and  after 
follow-on  testing.  During  correction  time,  development  teams  return  to  their  respective 
design  facilities  and  address  any  identified  shortfalls  before  LTA  execution. 

B.  LIMITED  TECHNICAL  ASSESSMENTS 

The  LTAs  serve  as  the  primary  setting  for  all  campaign  experimentation.  LTAs 
take  place  twice  annually  in  periods  of  five  business  days  in  order  to  ensure  that 
development  teams  have  the  time  to  acquire  and  build  the  technologies  needed.  The 
LTAs  incorporate  the  principles  of  variety  and  replication.  To  ensure  variety, 
experiments  rotate  which  components  of  the  system  are  being  tested.  To  ensure 
replication,  the  experimentation  environment  remains  constant.  During  later  LTAs, 
measures  met  during  prior  LTAs  must  be  repeated.  The  environment  for  LTAs  three 
through  six  is  a  relatively  flat,  outdoor  training  area  with  a  simulated  urban  setting.  This 
environment  will  allow  new  variables  to  be  introduced  and  reduces  mobility  challenges 
for  the  unmanned  system  at  an  early  development  stage.  As  the  system  matures  and  the 
capabilities  of  the  unmanned  systems  increase,  the  environment  must  change  to  present 
new,  realistic  challenges  to  the  system.  Future  environments  should  test  the  system’s 
functionality  during  increasingly  difficult  terrain  and  longer  distances. 

Appendix  E  is  a  modified  letter  of  instruction  provided  by  3D  Low  Altitude  Air 
Defense  Battalion.  The  letter  is  a  template  to  organize  and  focus  the  requirements  of 
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LTAs.  The  document  serves  as  an  easily  understandable  planning.  The  worksheet 
prompts  developers  to  explain  the  situation  and  intent  of  the  experimentation  and  LTA.  It 
also  prompts  developers  to  create  a  mission  statement  to  help  focus  the  efforts  of  the 
development  teams  involved  in  the  LTA.  Developers  also  identify  the  critical  operational 
issues  being  addressed  through  the  template.  The  critical  operational  issues  (COIs)  serve 
as  hypotheses  for  experimentation.  The  UTACC  team  has  created  seven  COIs  and  they 
are  disclosed  in  the  modified  template  in  the  appendix.  The  template  prompts  developers 
to  provide  a  concept  of  operations,  clearly  explaining  how  the  LTA  will  be  executed. 
Developers  should  use  phases  because  they  provide  clear  lines  of  delineation  between  the 
stages  of  experimentation.  Lastly,  the  template  requires  that  developers  create 
coordinating  instructions.  These  are  instructions  that  highlight  information  important  to 
all  teams,  such  as  entrance  and  exit  criteria  for  the  LTA  and  individual  phases.  Appendix 
E  is  filled  out  describing  the  recommended  execution  of  LTA  three. 

Developers  must  identify  entrance  criteria,  exit  criteria,  and  critical  operational 
issues  that  are  key  prior  to  LTA  execution.  Measures  of  effectiveness  and  measures  of 
performance  are  the  entrance  and  exit  criteria  for  UTACC  experimentation.  Appendix  C 
displays  MOEs  and  MOPs  developed  for  UTACC.  The  MOEs  and  MOPs  provide 
quantifiable  standards  to  evaluate  newly  developed  system  capabilities.  Using  them  as 
entrance  and  exit  criteria  ensures  that  replication  occurs  throughout  the  experimentation 
process  and  that  the  needs  of  the  Marine  Corps  are  met. 

In  conjunction  with  entrance  and  exit  criteria,  COIs  must  be  identified.  The 
Defense  Acquisition  University  defines  critical  operational  issues  as  “key  operational 
effectiveness  or  suitability  issues  that  must  be  examined  in  operational  test  and  evaluation 
to  determine  the  system’s  capability  to  perform  its  mission”  (Glossary,  n.d.).  Seven  COIs 
for  UTACC  are: 

1)  Will  the  system  reduce  the  cognitive  load  of  the  team? 

2)  Will  the  system  render  enhanced  3-dimensional  reconnaissance 
products? 

3)  Will  the  system  increase  the  safety  of  the  team? 

4)  Will  the  system  enhance  identification  and  engagement  of  targets? 
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5)  Does  the  system  operate  in  accordance  with  Marine  Corps 
doctrine? 

6)  To  what  extent  does  the  digital  plan  provide  context  to  the 
machines  as  well  as  the  Marines? 

7)  Does  the  system  demonstrate  flexibility  to  changes  in  the 
environment/plan? 

These  COIs  satisfy  the  requirement  for  all  LTAs  in  the  campaign  of  experimentation. 

C.  SUMMARY 

UTACC  is  a  unique  system  of  systems  that  enhances  mission  accomplishment  by 
reducing  the  Marines’  cognitive  load  in  combat.  It  does  this  through  increased  autonomy 
of  unmanned  ground  and  air  systems  and  improved  interdependence  between  human  and 
machine.  Taking  a  machine  from  tool  of  war  to  teammate  is  not  easily  accomplished.  A 
campaign  of  experimentation  facilitates  the  accomplishment  of  this  transition.  The 
campaign  of  experimentation  is  based  on  the  ideas  introduced  in  the  UTACC  CONOPs 
and  utilizes  the  MOEs  and  MOPs  (Appendix  C)  developed  in  the  fourth  thesis  of  the 
UTACC  series.  The  goal  of  the  campaign  is  to  create  an  incremental  plan  for 
experimentation  that  focuses  development  resources  and  the  attention  of  developers  to 
ensure  the  success  of  the  system  (Alberts  et  al.,  2006,  p.  63).  The  campaign  of 
experimentation  uses  the  guiding  principles  of  variety  and  replication  to  ensure  success. 
Variety  allows  developers  to  identify  variables  that  may  require  further  experimentation 
(Alberts  et  al.,  2006,  p.  65).  Replication  demonstrates  that  the  results  of  experimentation 
are  not  unique  to  a  specific  set  of  conditions  (Alberts  et  al.,  2006,  p.  65).  Creating  a 
balance  between  these  principles  facilitates  a  robust  series  of  experimentation  that 
advances  system  development  (Alberts  et  al.,  2006,  p.  64). 

The  campaign  of  experimentation  takes  advantage  of  limited  technical 
assessments  to  provide  the  setting  for  experimentation.  A  Gantt  chart  (Appendix  D) 
outlines  the  timeline,  link,  and  subject  of  experimentation  for  each  of  the  LTAs.  With 
funding  for  the  project  guaranteed  until  2019,  the  campaign  of  experimentation  covers 
that  time  period.  Two  LTAs  a  year  will  provide  developers  the  time  needed  to  acquire 
technologies  and  conduct  functionality  testing  on  their  own.  The  LTAs  integrate  coactive 
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design  and  model  driven  software  development  as  preferred  design  methodologies. 
The  LTA  schedule  also  integrates  acceptance  and  follow-on  testing.  These  testing 
periods  are  important  for  the  success  of  experimentation  taking  place  during  the  LTAs. 
With  acceptance  and  follow-on  testing,  MCWL  can  assess  system  development  and 
technologies,  identify  capability  gaps,  and  improve  coordination  between  development 
teams  prior  to  the  LTA. 

The  LTA  schedule  integrates  variety  and  replication.  Experimenting  with 
different  technologies  during  each  LTA  satisfies  the  requirement  for  variety.  Experiments 
meet  the  requirement  for  replication  when  each  LTA  uses  the  same  environment  and  each 
LTA  repeats  the  accomplishments  the  prior  LTA  .  The  MOEs  and  MOPs  provide  the 
performance  standards  and  entrance  and  exit  criteria  for  each  LTA.  The  environment  for 
each  LTA  is  a  relatively  flat,  outdoor  training  area  with  a  simulated  urban  setting.  The 
same  training  area  should  be  used  for  each  LTA  if  possible.  Appendix  E  provides  a 
worksheet  template  to  facilitate  planning  for  the  LTAs.  The  worksheet  is  a  modified  letter 
of  instruction  used  by  Marine  Corps  field  units  to  communicate  how  the  execution  of  an 
event,  such  as  a  field  exercise,  will  occur.  A  unique  requirement  of  the  worksheet  is  that 
critical  operational  issues  are  identified.  This  thesis  provides  seven  COIs  that  can  remain 
consistent  across  all  LTA  efforts.  The  COIs  provided  address  the  breadth  of  the  UTACC 
system  as  it  currently  exists. 

D.  RECOMMENDATIONS 

This  thesis  plans  for  how  future  experimentation  should  occur.  Going  forward,  the 
campaign  of  experimentation  should  be  modified  to  incorporate  future  areas  of 
experimentation  so  that  future  system  development  is  incremental.  UTACC’ s  success 
depends  upon  further  experimentation  on  interdependence  and  the  planning  capability  of 
the  system.  Interdependence  enhances  the  system’s  ability  to  serve  as  a  team  member. 
The  smooth  push  and  pull  of  information  puts  the  Marine  on  the  loop  with  the  system.  On 
the  loop  status  reduces  Marines’  cognitive  load  and  increases  mission  accomplishment. 
Specifically,  experiments  testing  different  interfaces  between  the  human  and  machine  are 
essential  to  UTACC  success. 
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Experiments  focused  on  integrating  information  into  a  plan  are  critical  for  the 
system  to  be  successful  in  the  field.  The  system  must  be  capable  of  taking  on  and 
simplifying  tasks  carried  out  by  Marines  to  be  successful.  Planning  requires  time,  energy, 
and  resources  and  can  take  place  during  any  stage  of  a  mission.  Spontaneous  mission 
events  require  planning  updates.  UTACC’s  ability  to  collect  information  on  the  terrain,  to 
identify  potential  threats,  and  to  incorporate  threats  and  terrain  into  a  plan  for  a  team 
leader  is  invaluable.  The  system  is  merely  suggesting  a  plan;  the  ultimate  decision  to 
accept,  reject,  or  modify  the  plan  rests  with  the  Marine.  Regardless  of  the  team  leader’s 
decision,  having  critical  information  consolidated  and  organized  facilitates  rapid  planning 
and  execution  for  Marines.  With  future  experimentation,  UTACC  will  evolve  from  a 
machine  to  a  functioning  Marine  teammate. 
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APPENDIX  A.  BAMCIS  MODEL 
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APPENDIX  B.  INTERDEPENDENCE  ANALYSIS  TABLE 


Option  1  |  Option  2  |  Option  3 


BAMCIS 

STEP 

Tasks 

Subtasks 

Capacities 

U 

A 

S 

u 

G 

S 

M 

U 

A 

S 

M 

U 

A 

S 

u 

G 

S 

OPD 

requirements 

Troop 

Leading 

Step 

(A) 

Main 

Task 

(A.  1) 

Subtask  of 
Main 
Task  (A) 

(A.  1.1) 
Capacity 
required  for 
(A.1) 

Mechanisms, 
interface  design 
elements,  etc.  that 
meet  the 
Observability, 
Predictability, 
Directability 
requirements 
synthesized 
through  the 
analysis  of  the 
interdependent 
teaming  role 
alternatives. 

(A.  1.2) 
Capacity 
required  for 
(A.1) 

(A.2) 

Subtask  of 
Main 
Task  (A) 

(A.2.1) 
Capacity 
required  for 
(A.2) 

(A.3) 

Subtask  of 
Main 
Task  (A) 

(A.3.1) 
Capacity 
required  for 
(A.3) 

(B) 

Main 

Task 

(B.l) 

Subtask  of 
Main 
Task  (B) 

(B.  1.1) 
Capacity 
required  for 

(B.l) 

(B.2) 

Subtask  of 
Main 
Task  (B) 

(B.2.1) 
Capacity 
required  for 
(B.2) 

(B.2.2) 

Capacity 
required  for 

(B.2) 

39 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


40 


APPENDIX  C.  MOE  AND  MOP  EXAMPLE 


Case 

Priority 

Objective 

MCT 

MCT 

Description 

MOP 

Result 

Unit 

Description 

1 

High 

Jointly 

Produce 

Map 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

Ml 

75 

% 

Input  Orientation:  Upload  the 
present  location,  direction  of  attack 
and  objective,  and  known  key 
terrain  data 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M2 

80 

% 

Situation:  Contains  information  on 
enemy  (which  will  include  SALUTE, 
DRAW-D,  EMLCOA  and  EMDCOA) 
and  friendly  (which  includes 
locations  and  missions  of  higher, 
adjacent  and  supporting  units) 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M3 

55 

% 

Mission:  Upload  the  UXV's  mission 
as  related  to  the  mission  of  the 
team  (Who,  What,  When,  Where, 
Why).  Include  tactical  tasks. 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M4 

60 

% 

Execution:  Upload  Concept  of 
Operations  (Commander's  Intent, 
Scheme  of  Maneuver,  Fire  Support 
Plan),  Tasks  and  Coordinating 
Instructions 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M5 

67 

% 

Admin  and  Logistics:  Define 
number  and  roles  of  humans  and 
robots  collaborating  in  team 
environment,  and  establish 
refueling  and  RTB  points  if  different 
from  origin 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 
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2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

UTAC 

C  2.1 

Conduct  Initial 
Mapping  - 
Depart 

Friendly  Lines 

Ml 

Y 

Y/N 

Resolve  airspace  deconfliction  and 
meet  safety  threshold  for  launch. 

UTAC 

C  2.1 

Conduct  Initial 
Mapping  -  Geo 
Scan 

M2 

2 

Hrs 

Understand  the  size  of  area  to  scan 
between  origin  and  objective.  Scan 
the  area  between  origin  and 
objective  for  specific  geographic 
features.  Scan  objective  area  for 
basic  geography.  Execute  mapping 
protocol.  Generate  actionable 
information. 

UTAC 

Conduct  Initial 

M3 

1.5 

Hrs 

Transmit  map  info,  identify  urban 
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C  2.1 

Mapping  - 
Build  Map 

and  wooded  areas,  identify  masked 
areas,  and  fill  in  gaps  in  intel. 

UTAC 

C  2.2 

Select 

Emphasis  Area 
-  Review  Map 

Ml 

0.5 

Hrs 

Different  angle,  higher  resolution, 
different  sensor,  camera  direction, 
multiple  directions.  Identify 
potential  danger  areas,  routes,  LZ's, 
water  features. ..etc. 

2.2.1. 

1 

Conduct  Route 

Reconnaissanc 

e 

M4 

1 

Hrs 

To  conduct  initial  route  study 
(dismounted/mounted). 

UTAC 

C  3.1 

Conduct 

Detailed 

Mapping 

Ml 

70 

% 

Scan  Emphasis  Areas.  Execute 
detailed  mapping  protocol  (the 
protocol  will  be  different  for  why 
we  selected  the  area  for  additional 
emphasis)  i.e.  If  for  LZ,  execute  the 
LZ  protocol,  if  for  route  then  etc. 
Build  detailed  map  collaboratively. 

UTAC 

C  3.2 

MCOO 

M2 

25 

% 

Depict  Surface  Drainage.  Depict 
water  sources  (width,  depth, 
velocity,  bank  slope,  height,  and 
potential  flood  zones) 

2.2.1. 

1 

Conduct  Route 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Route/road  confirmed. 

1.5 

High 

Jointly 

Produce 

Map  of 
Alternate 

Environment 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

Ml 

75 

% 

Input  Orientation:  Upload  the 
present  location,  direction  of  attack 
and  objective,  and  known  key 
terrain  data 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M2 

80 

% 

Situation:  Contains  information  on 
enemy  (which  will  include  SALUTE, 
DRAW-D,  EMLCOA  and  EMDCOA) 
and  friendly  (which  includes 
locations  and  missions  of  higher, 
adjacent  and  supporting  units) 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M3 

55 

% 

Mission:  Upload  the  UxV's  mission 
as  related  to  the  mission  of  the 
team  (Who,  What,  When,  Where, 
Why).  Include  tactical  tasks. 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M4 

60 

% 

Execution:  Upload  Concept  of 
Operations  (Commander's  Intent, 
Scheme  of  Maneuver,  Fire  Support 
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Plan),  Tasks  and  Coordinating 
Instructions 

UTAC 

C  1.2 

Enter  Mission 

Parameters 

M5 

67 

% 

Admin  and  Logistics:  Define 
number  and  roles  of  humans  and 
robots  collaborating  in  team 
environment,  and  establish 
refueling  and  RTB  points  if  different 
from  origin 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 
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2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

UTAC 

C  2.1 

Conduct  Initial 
Mapping  - 
Depart 

Friendly  Lines 

Ml 

Y 

Y/N 

Resolve  airspace  de-confliction  and 
meet  safety  threshold  for  launch. 

UTAC 

C  2.1 

Conduct  Initial 
Mapping  -  Geo 
Scan 

M2 

2 

Hrs 

Understand  the  size  of  area  to  scan 
between  origin  and  objective.  Scan 
the  area  between  origin  and 
objective  for  specific  geographic 
features.  Scan  objective  area 
for  basic  geography.  Execute 
mapping  protocol.  Generate 
actionable  information. 

UTAC 

C  2.1 

Conduct  Initial 
Mapping  - 
Build  Map 

M3 

1.5 

Hrs 

Transmit  map  info,  identify  urban 
and  wooded  areas,  identify  masked 
areas,  and  fill  in  gaps  in  intel. 

UTAC 

C  2.2 

Select 

Emphasis  Area 
-  Review  Map 

Ml 

0.5 

Hrs 

Different  angle,  higher  resolution, 
different  sensor,  camera  direction, 
multiple  directions.  Identify 
potential  danger  areas,  routes,  LZ's, 
water  features. ..etc. 

2.2.1. 

1 

Conduct  Route 

Reconnaissanc 

e 

M4 

1 

Hrs 

To  conduct  initial  route  study 
(dismounted/mounted). 

UTAC 

C  3.1 

Conduct 

Detailed 

Mapping 

Ml 

70 

% 

Scan  Emphasis  Areas.  Execute 
detailed  mapping  protocol  (the 
protocol  will  be  different  for  why 
we  selected  the  area  for  additional 
emphasis)  i.e.  If  for  LZ,  execute  the 

LZ  protocol,  if  for  route  then  etc. 
Build  detailed  map  collaboratively. 

UTAC 

C  3.2 

MCOO 

M2 

25 

% 

Depict  Surface  Drainage.  Depict 
water  sources  (width,  depth, 
velocity,  bank  slope,  height,  and 
potential  flood  zones) 

2.2.1. 

1 

Conduct  Route 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Route/road  confirmed. 

2 

High 

Target  Only 
Visible  to 

UGV 

2.2.1. 

Conduct  Area 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
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2 

Reconnaissanc 

e 

reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

M2 

25 

% 

Of  targets  accurately  located. 
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and 

Intelligence 

3 

High 

Target  Only 
Visible  to 

UAV 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 
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e  and 

Surveillance 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 

4 

High 

Target  Not 
Present 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 
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e  and 

Surveillance 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 

4.5 

Low 

Evasive 

Target 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
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infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 

5 

High 

Only 

Incorrect 

Target(s) 

Present 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
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Reconnaissanc 

e 

line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 

6 

High 

Both  Correct 

and  Incorrect 

Targets 

Present 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 
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e 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 

8 

High 

Start  Hunt 
for  Target  at 
Suspected 
Location 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

Ml 

0.2 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 

2.2.1. 

2 

Conduct  Area 

Reconnaissanc 

e 

M2 

Y 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  to  the  Commander  and 

staff. 

2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

Ml 

0.5 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance  assets  in  place. 
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2.2.1. 

3 

Conduct  Zone 

Reconnaissanc 

e 

M2 

N 

Y/N 

Provide  photographic  and 
descriptive  data  of  the  Named  Area 
of  Interest  (NAI)  to  the  Commander 
and  staff. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M3 

34 

% 

Of  equipment  ready  and  available 
to  provide  air  reconnaissance 
operations. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M4 

Y 

Y/N 

Product  (sensor) 

dissemination/distribution  network 
available. 

2.2.5. 

2 

Conduct 

Aviation 

Reconnaissanc 

e 

M7 

N 

Y/N 

Able  to  communicate  relevant 
reconnaissance  information  using 
line-of-site  (LOS)/beyond-line-of- 
site  (BLOS)  means. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M2 

45 

% 

Of  equipment  ready  and  available 
to  provide  reconnaissance  and 
surveillance  operations  (i.e., 
communications,  target 
designation,  crew  served  weapons, 
infiltration/exfiltration  equipment, 
mobility  assets). 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M4 

1 

Hrs 

From  receipt  of  tasking,  unit 
reconnaissance/surveillance  assets 
in  place. 

2.7 

Conduct 

Ground 

Reconnaissanc 

e  and 

Surveillance 

M5 

70 

% 

Of  collection  requirements  fulfilled 
by  reconnaissance/surveillance 
assets. 

2.2 

Collect  Data 

and 

Intelligence 

Ml 

25 

% 

Of  targets  accurately  identified. 

2.2 

Collect  Data 

and 

Intelligence 

M2 

25 

% 

Of  targets  accurately  located. 
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APPENDIX  D.  GANTT  CHART  OUTLINE 


Task  Name 

▼ 

Duration 

▼ 

Start 

▼ 

Finish 

▼ 

LTA  2 

5  days 

Mon  4/18/16 

Fri  4/22/16 

”  LTA  3  Preparation 

255  days 

Mon  4/25/16 

Fri  4/14/17 

OutdoorTransition/ Software  Development 

205  days 

Mon  4/25/16 

Fri  2/3/17 

Acceptance  Testing 

5  days 

Mon  2/6/17 

Fri  2/10/17 

Correction  Time 

15  days 

Mon  2/13/17 

Fri  3/3/17 

Follow-  on  Acceptance  Testing 

5  days 

Mon  3/6/17 

Fri  3/10/17 

Correction  Time 

25  days 

Mon  3/13/17 

Fri  4/14/17 

LTA  3 

5  days 

Mon  4/17/17 

Fri  4/21/17 

~  LTA  4  Preparation 

130  days 

Mon  4/24/17 

Fri  10/20/17 

Coactive  Design 

30  days 

Mon  4/24/17 

Fri  6/2/17 

Model  Driven  Software  Development  (MDSD) 

30  days 

Mon  4/24/17 

Fri  6/2/17 

Voice  Command  Recognition 

50  days 

Mon  6/5/17 

Fri  8/11/17 

Acceptance  Testing 

5  days 

Mon  8/14/17 

Fri  8/18/17 

Correction  Time 

25  days 

Mon  8/21/17 

Fri  9/22/17 

Follow-  on  Acceptance  Testing 

5  days 

Mon  9/25/17 

Fri  9/29/17 

Correction  Time 

15  days 

Mon  10/2/17 

Fri  10/20/17 

LTA  4 

5  days 

Mon  10/23/17 

Fri  10/27/17 

”  LTA  5  Preparation 

125  days 

Mon  10/30/17 

Fri  4/20/18 

Coactive  Design/MDSD 

30  days 

Mon  10/30/17 

Fri  12/8/17 

Auto-Follow  Development 

45  days 

Mon  12/11/17 

Fri  2/9/18 

Hand  and  Arm  Signal  Recognition 

45  days 

Mon  12/11/17 

Fri  2/9/18 

Acceptance  Testing 

5  days 

Mon  2/12/18 

Fri  2/16/18 

Correction  Time 

21  days 

Fri  2/16/18 

Fri  3/16/18 

Follow-  on  Acceptance  Testing 

5  days 

Mon  3/19/18 

Fri  3/23/18 

Correction  Time 

20  days 

Mon  3/26/18 

Fri  4/20/18 

LTA  5 

5  days 

Mon  4/23/18 

Fri  4/27/18 

'  LTA  6  Preparation 

125  days 

Mon  4/30/18 

Fri  10/19/18 

Coactive  Design/MDSD 

30  days 

Mon  4/30/18 

Fri  6/8/18 

Threat  Analysis 

45  days 

Mon  6/11/18 

Fri  8/10/18 

Self  Diagnostic 

45  days 

Mon  6/11/18 

Fri  8/10/18 

Acceptance  Testing 

5  days 

Mon  8/13/18 

Fri  8/17/18 

Correction  Time 

20  days 

Mon  8/20/18 

Fri  9/14/18 

Follow-  on  Acceptance  Testing 

5  days 

Mon  9/17/18 

Fri  9/21/18 

Correction  Time 

20  days 

Mon  9/24/18 

Fri  10/19/18 

LTA  6 

5  days 

Mon  10/22/18 

Fri  10/26/18 
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APPENDIX  E.  LTA  WORKSHEET 


UTACC  LTA  3 

May  2016 

References 

(1)  COBP  Campaigns  of  Experimentation 

(2)  COBP  Experimentation 

(3)  LTA  2  Scenario  Tasks 

Enclosures 

(1)  MCT  List 

Task  org: 

Marine  Corps  Warfighting  Lab 

SITUATION:  Marine  Corps  Warfighting  Lab  (MCWL)  has  tasked  Naval 

Postgraduate  School  (NPS)  and  industry  partners  with  the  development 
of  the  Unmanned  Tactical  Autonomous  Control  and  Collaboration  system. 
To  accomplish  this  task  a  series  of  Limited  Technical  Assessments 
(LTA)  and  Limited  Objective  Experiments  (LOE)  are  needed  to  create  a 
viable  system  that  meets  the  requirements  of  MCWL  and  enhance  the 
warfighting  ability  of  the  Marine  Corps  as  a  whole.  These  LTAs  and 
LOEs  will  follow  the  fundamental  concepts  of  variety  and  replication 
as  put  forward  in  the  Code  of  Best  Practices  Campaigns  of 
Experimentation . 


MISSION:  NLT  17  April  2017, 

MCWL  sponsors  LTA  3,  location 
to  be  determined  in  order  to 
replicate  LTA  2  performance 
accomplishments,  and  advance 
the  UTACC  system  in  an  outdoor 
environment . 


Intent : 

1.  Purpose.  The  purpose  of  LTA  3  is 
to  take  the  UTACC  system  from  an  indoor 
controlled  environment  to  an  outdoor 
controlled  environment  while  further 
testing  the  capabilities  of  the  system 
interface,  onboard  sensors,  and 
software  as  well  as  new  robotic 
platforms . 


2.  Method.  Having  met  the 
requirements  for  a)  UTACC  software 
utilization  in  the  GUSS  autonomous 
system  and  Phoenix  UAV,  and  b) 
demonstration  of  successful  outdoor 
transition  in  acceptance  and  follow-on 
testing,  LTA  3  will  be  conducted  in  an 
outdoor  environment  with  a  simulated 
urban  setting.  This  venue  will  allow 
the  system  developers  to  replicate 
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COI : 

1)  Will  the  system  reduce 
the  cognitive  load  of 
the  team? 

2)  Will  the  system  render 
enhanced  3-Dimensional 
reconnaissance  products 

3)  Will  the  system  increas 
the  safety  of  the  team? 

4)  Will  the  system  enhance 
identification  and 
engagement  of  targets? 

5)  Does  the  system  operate 
in  accordance  with 
Marine  Corps  doctrine? 

6)  To  what  extent  does  the 
digital  plan  provide 
context  to  the  machines 
as  well  as  the  Marines? 

7)  Does  the  system 
demonstrate  flexibility 
to  changes  in  the 
environment /plan? 


EXECUTION: 

Concept  of  Operations : 

1.  On  17  April  2017,  required  personnel  involved  in  the  UTACC 
system  development  will  arrive  at  a  testing  and  evaluation 
location  selected  by  MCWL .  The  desired  location  for  testing  and 
evaluation  is  a  relatively  flat,  outdoor  training  area  with  a 
simulated  urban  setting.  This  environment  will  allow  the  system 
to  be  newly  introduced  to  an  outdoor  environment  that  will  not 
provide  unnecessary  mobility  challenges  at  such  an  early  stage 
of  development.  This  environment  will  simultaneously  provide  a 
setting  that  facilitates  the  advancement  and  replication  of 
previous  LTA  accomplishments.  The  environment  will  also 
facilitate  testing  of  the  capabilities  of  the  GUSS  autonomous 
vehicle  and  Phoenix  90  UAV  under  the  control  of  UTACC  software 
and  interfaces . 

2.  Testing  will  be  broken  into  three  phases.  Phase  I  will  begin 
with  a  Marine  operator  inputting  mission  parameters  and 
releasing  the  system  to  begin  reconnaissance.  The  system  will 
conduct  a  reconnaissance  of  an  identified  area  of  interest 
(AOI),  building  a  3D  map  as  it  does  so.  Upon  recognizing  that 
it  cannot  complete  the  mission  as  required,  the  system  alerts 
the  Marine  and  the  Marine  authorizes  the  launch  of  the  systems 
UAV  to  assist  in  the  completion  of  the  mission.  When 
reconnaissance  is  completed,  the  system  requests  permission  from 
the  Marine  to  return  to  base  (RTB) .  When  the  Marine  operator 
gives  authorization,  the  system  will  return  to  its  original 
start  point.  Exit  criteria  for  Phase  I  will  be  that  all 
performance  standards  established  in  MCTs  1,  1.5,  and  2  are  met. 


results  from  LTA  2-  specifically  MCTs 
1,  1.5,  and  2-  while  advancing  the 
capabilities  demonstrated  in  these  MCTs 
in  an  outdoor  environment.  LTA  3  will 
be  conducted  in  three  phases  utilizing 
the  GUSS  autonomous  vehicle  and  Phoenix 
90  UAV  as  robotic  platforms.  The 
system  will  be  tasked  with  creating  a 
3D  model  of  the  environment,  facial 
recognition  of  a  person  of  interest, 
reaction  to  a  newly  introduced  variable 
in  the  environment,  and 
deriving/building  a  digital  plan  for 
the  approach  to  the  target. 

3.  End  state.  The  UTACC  system 
demonstrates  the  ability  to  meet 
previously  tested  MCT  performance 
requirements  in  an  outdoor  environment 
while  identifying  shortfalls/  advancing 
the  capabilities  of  the  system 
interface,  onboard  sensors,  and 
software . 
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3.  Phase  II  will  replicate  the  results  of  Phase  I  with  the  addition 
of  the  system  being  required  to  conduct  facial  recognition  of  a 
person  and/or  object  of  interest.  After  receiving  verification 
from  the  Marine  operator  that  the  system  has  accurately 
recognized  the  target,  the  system  will  request  permission  to 
engage.  When  the  Marine  operator  authorizes  engagement,  the 
system  will  relay  a  request  for  fire  to  a  "firing  element." 

Exit  criteria  for  Phase  II  are  that  previous  MCTs  are  met  and 
successful  target  recognition  and  engagement  has  been  completed. 

4.  Phase  III  will  begin  at  the  completion  of  Phase  II  and  will 
accomplish  the  performance  goals  of  Phases  I  and  II  while 
reacting  effectively  to  the  introduction  of  a  new  variable  and 
demonstrating  basic  planning  capabilities.  Utilizing  the 
currently  generated  3D  map  of  the  AOI,  the  system  will  be 
required  a  deliver  a  plan  for  approval,  disapproval,  or 
modification  to  the  Marine  operator.  The  plan  will  orient  the 
Marine  to  the  AOI  and  suggest  a  potential  approach  route  to  an 
operator  selected  waypoint.  Also  in  this  phase,  a  vehicle, 
previously  uploaded  to  the  system  as  a  BOLO,  will  enter  the  AOI. 
The  system  must  accurately  identify  the  vehicle  as  the  BOLO 
vehicle,  notify  the  Marine  operator  for  verification,  and 
request  guidance  for  follow-on  action.  Follow-on  action  can 
consist  of  targeting,  observation  utilizing  either  the  UGV  or 
UAV  (the  decision  for  asset  usage  will  be  left  to  the  Marine 
operator) ,  or  to  ignore  the  vehicle  and  continue  with  previous 
mission  tasking. 


5.  Phase  III  exit  criteria  are  as  follows; 


a.  Successful  completion  of  MCTs  1,  1.5,  and  2, 

b.  Successful  facial  recognition  and  engagement  of  a  target, 

c.  Successful  identification  of  a  newly  introduced  BOLO 
vehicle  with  requests  for  action;  targeting,  observation 
utilizing  either  the  UGV  or  UAV,  or  to  ignore  the  vehicle 
and  continue  with  previous  mission  tasking, 

d.  A  mission  plan,  created  from  the  3D  map,  utilizing  a 
Marine  operator  selected  waypoint  is  successfully 
generated  for  approval,  disapproval,  or  modification. 


6.  LTA  3  is  complete  when  all  three  phases  and  their  associated 
entrance  and  exit  criteria  are  met. 
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